Database Module
The db module provides comprehensive SQL database connectivity with connection pooling, transaction support, and GORM-inspired ORM features. It supports SQLite3, PostgreSQL, and MySQL databases with SQLC-like query workflows.
Features
- ✅ Multiple Database Support: SQLite3, PostgreSQL, MySQL
- ✅ Connection Pooling: Configurable pool size and connection lifetime
- ✅ SQLC-style Queries: Type-safe query execution with parameter binding
- ✅ Transaction Support: Manual and automatic transaction management
- ✅ GORM-inspired ORM: Simple CRUD operations with map-based data
- ✅ Map Integration: Results returned as Harneet maps
- ✅ JSON Ready: Seamless JSON serialization of query results
Import
Connection Management
Open(driver string, dsn string) (connection, error)
Opens a database connection with the specified driver and data source name.
Supported Drivers: - sqlite3 - SQLite file-based database - postgres - PostgreSQL database - mysql - MySQL database
Example:
| Open Connection Example |
|---|
| import db
import fmt
// SQLite3
var conn, err = db.Open("sqlite3", "myapp.db")
// PostgreSQL
var conn, err = db.Open("postgres", "host=localhost user=postgres password=secret dbname=mydb sslmode=disable")
// MySQL
var conn, err = db.Open("mysql", "user:password@tcp(localhost:3306)/dbname")
if err != None {
fmt.Printf("Connection failed: %v\n", err)
return
}
|
Close(conn connection) error
Closes the database connection and releases resources.
| Close Connection |
|---|
| var err = db.Close(conn)
if err != None {
fmt.Printf("Close error: %v\n", err)
}
|
Ping(conn connection) error
Verifies the database connection is alive.
| Ping Connection |
|---|
| var err = db.Ping(conn)
if err != None {
fmt.Println("Database connection lost")
}
|
Connection Pool Configuration
SetMaxOpenConns(conn connection, n int) None
Sets the maximum number of open connections to the database.
| SetMaxOpenConns |
|---|
| db.SetMaxOpenConns(conn, 25) // Max 25 connections
|
SetMaxIdleConns(conn connection, n int) None
Sets the maximum number of idle connections in the pool.
| SetMaxIdleConns |
|---|
| db.SetMaxIdleConns(conn, 5) // Keep 5 idle connections
|
SetConnMaxLifetime(conn connection, seconds int) None
Sets the maximum lifetime of a connection in seconds.
| SetConnMaxLifetime |
|---|
| db.SetConnMaxLifetime(conn, 300) // 5 minutes max lifetime
|
Query Execution (SQLC-style)
Query(conn connection, sql string, ...args) (array[map], error)
Executes a SQL query that returns multiple rows. Results are returned as an array of maps where each map represents a row.
| Query Example |
|---|
| // Query with no parameters
var users, err = db.Query(conn, "SELECT * FROM users")
// Query with parameters (uses ? placeholders)
var adults, err = db.Query(conn, "SELECT * FROM users WHERE age >= ?", 18)
// Multiple parameters
var results, err = db.Query(conn,
"SELECT * FROM users WHERE age >= ? AND city = ?",
18, "New York")
if err == None {
for row in results {
fmt.Printf("ID: %d, Name: %s, Age: %d\n",
row["id"], row["name"], row["age"])
}
}
|
QueryRow(conn connection, sql string, ...args) (map, error)
Executes a SQL query that returns a single row.
| QueryRow Example |
|---|
| var user, err = db.QueryRow(conn, "SELECT * FROM users WHERE id = ?", 1)
if err != None {
fmt.Println("User not found")
return
}
fmt.Printf("User: %s, Email: %s\n", user["name"], user["email"])
|
Exec(conn connection, sql string, ...args) (result, error)
Executes a SQL statement (INSERT, UPDATE, DELETE) that doesn't return rows.
| Exec Example |
|---|
| // INSERT
var result, err = db.Exec(conn,
"INSERT INTO users (name, email, age) VALUES (?, ?, ?)",
"Alice", "alice@example.com", 30)
// UPDATE
var result, err = db.Exec(conn,
"UPDATE users SET age = ? WHERE id = ?",
31, 1)
// DELETE
var result, err = db.Exec(conn,
"DELETE FROM users WHERE age < ?",
18)
|
LastInsertId(result) (int, error)
Returns the last inserted ID from an INSERT operation.
| LastInsertId Example |
|---|
| var result, err = db.Exec(conn, "INSERT INTO users (name) VALUES (?)", "Bob")
if err == None {
var id, _ = db.LastInsertId(result)
fmt.Printf("New user ID: %d\n", id)
}
|
RowsAffected(result) (int, error)
Returns the number of rows affected by an UPDATE or DELETE operation.
| RowsAffected Example |
|---|
| var result, err = db.Exec(conn, "UPDATE users SET active = 1")
if err == None {
var count, _ = db.RowsAffected(result)
fmt.Printf("Updated %d users\n", count)
}
|
Transaction Support
Begin(conn connection) (transaction, error)
Starts a new database transaction.
| Begin Transaction |
|---|
| var tx, err = db.Begin(conn)
if err != None {
fmt.Println("Failed to start transaction")
return
}
|
Commit(tx transaction) error
Commits the current transaction.
| Commit Transaction |
|---|
| var err = db.Commit(tx)
if err != None {
fmt.Println("Commit failed")
}
|
Rollback(tx transaction) error
Rolls back the current transaction.
| Rollback Transaction |
|---|
| var err = db.Rollback(tx)
if err != None {
fmt.Println("Rollback failed")
}
|
TxQuery(tx transaction, sql string, ...args) (array[map], error)
Executes a query within a transaction.
| TxQuery Example |
|---|
| var tx, _ = db.Begin(conn)
var users, err = db.TxQuery(tx, "SELECT * FROM users WHERE age > ?", 18)
|
TxExec(tx transaction, sql string, ...args) (result, error)
Executes a statement within a transaction.
| TxExec Example |
|---|
| var tx, _ = db.Begin(conn)
var result, err = db.TxExec(tx, "INSERT INTO users (name) VALUES (?)", "Charlie")
|
Transaction Example
| Transaction Example |
|---|
| import db
import fmt
function transferMoney(conn any, fromId int, toId int, amount int) error {
// Start transaction
var tx, err = db.Begin(conn)
if err != None {
return err
}
// Deduct from sender
var result1, err1 = db.TxExec(tx,
"UPDATE accounts SET balance = balance - ? WHERE user_id = ?",
amount, fromId)
// Add to receiver
var result2, err2 = db.TxExec(tx,
"UPDATE accounts SET balance = balance + ? WHERE user_id = ?",
amount, toId)
// Check if both succeeded
if err1 != None or err2 != None {
db.Rollback(tx)
fmt.Println("Transaction rolled back")
return err1
}
// Commit transaction
var commitErr = db.Commit(tx)
if commitErr != None {
db.Rollback(tx)
return commitErr
}
fmt.Println("Transfer successful")
return None
}
|
ORM Features (GORM-inspired)
Create(conn connection, table string, data map) (int, error)
Inserts a new record into the database. Returns the inserted ID.
| Insert Example |
|---|
| var userData = {
"name": "Alice",
"email": "alice@example.com",
"age": 30
}
var id, err = db.Create(conn, "users", userData)
if err == None {
fmt.Printf("Created user with ID: %d\n", id)
}
|
Find(conn connection, table string, conditions map) (array[map], error)
Retrieves records from the database matching the conditions.
| Find Example |
|---|
| // Find all users aged 30
var users, err = db.Find(conn, "users", {"age": 30})
// Find users in a specific city
var users, err = db.Find(conn, "users", {"city": "New York"})
// Find with multiple conditions (AND)
var users, err = db.Find(conn, "users", {"age": 30, "city": "New York"})
if err == None {
for user in users {
fmt.Printf("User: %s\n", user["name"])
}
}
|
FindOne(conn connection, table string, conditions map) (map, error)
Retrieves a single record from the database.
| FindOne Example |
|---|
| var user, err = db.FindOne(conn, "users", {"id": 1})
if err != None {
fmt.Println("User not found")
return
}
fmt.Printf("Found: %s\n", user["name"])
|
Update(conn connection, table string, conditions map, data map) (int, error)
Updates records in the database. Returns the number of rows affected.
| Update Example |
|---|
| // Update specific user
var count, err = db.Update(conn, "users",
{"id": 1},
{"age": 31, "city": "Boston"})
// Update multiple users
var count, err = db.Update(conn, "users",
{"city": "New York"},
{"timezone": "EST"})
if err == None {
fmt.Printf("Updated %d records\n", count)
}
|
Delete(conn connection, table string, conditions map) (int, error)
Deletes records from the database. Returns the number of rows deleted.
Safety: Conditions map is required to prevent accidental deletion of all records.
| Delete Example |
|---|
| // Delete specific user
var count, err = db.Delete(conn, "users", {"id": 1})
// Delete multiple users
var count, err = db.Delete(conn, "users", {"age": 0})
if err == None {
fmt.Printf("Deleted %d records\n", count)
}
|
Count(conn connection, table string, conditions map) (int, error)
Counts records in the database matching the conditions.
| Count Example |
|---|
| // Count all users
var total, err = db.Count(conn, "users", {})
// Count users with conditions
var adults, err = db.Count(conn, "users", {"age": 18})
if err == None {
fmt.Printf("Found %d users\n", total)
}
|
Database Migrations
The database module includes comprehensive migration support with both up (apply) and down (rollback) capabilities, similar to Rails migrations or Flyway.
Migration Functions
MigrateUp(conn connection, version int, name string, upSQL string) error
Applies a migration to the database. Migrations are tracked in a schema_migrations table.
| MigrateUp Example |
|---|
| var err = db.MigrateUp(conn, 1, "create_users_table",
"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT, email TEXT)")
|
MigrateDown(conn connection, version int, downSQL string) error
Rolls back a migration. Only applies if the migration was previously applied.
| MigrateDown Example |
|---|
| var err = db.MigrateDown(conn, 1, "DROP TABLE users")
|
GetMigrationVersion(conn connection) (int, error)
Returns the current migration version number.
| GetMigrationVersion Example |
|---|
| var version, err = db.GetMigrationVersion(conn)
fmt.Printf("Current version: %d\n", version)
|
GetMigrationHistory(conn connection) (array[map], error)
Returns all applied migrations with version, name, and timestamp.
| GetMigrationHistory Example |
|---|
| var history, err = db.GetMigrationHistory(conn)
for migration in history {
fmt.Printf("v%d: %s (applied at: %d)\n",
migration["version"],
migration["name"],
migration["applied_at"])
}
|
MigrateToVersion(conn connection, targetVersion int, migrations array) error
Migrates to a specific version (up or down automatically). Migrations array should contain migration maps.
| MigrateTo Example |
|---|
| var migrations = [
{
"version": 1,
"name": "create_users",
"up": "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)",
"down": "DROP TABLE users"
},
{
"version": 2,
"name": "add_users_email",
"up": "ALTER TABLE users ADD COLUMN email TEXT",
"down": "ALTER TABLE users DROP COLUMN email"
}
]
// Migrate to version 2
var err = db.MigrateToVersion(conn, 2, migrations)
// Rollback to version 1
var err = db.MigrateToVersion(conn, 1, migrations)
|
Migration Example
| Migration Example |
|---|
| import db
import fmt
function main() {
var conn, _ = db.Open("sqlite3", "myapp.db")
// Define migrations
var migrations = [
{
"version": 1,
"name": "create_users_table",
"up": "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL, email TEXT UNIQUE)",
"down": "DROP TABLE users"
},
{
"version": 2,
"name": "add_users_age",
"up": "ALTER TABLE users ADD COLUMN age INTEGER",
"down": "ALTER TABLE users DROP COLUMN age"
},
{
"version": 3,
"name": "create_posts_table",
"up": "CREATE TABLE posts (id INTEGER PRIMARY KEY, user_id INTEGER, title TEXT, content TEXT, FOREIGN KEY(user_id) REFERENCES users(id))",
"down": "DROP TABLE posts"
}
]
// Check current version
var currentVersion, _ = db.GetMigrationVersion(conn)
fmt.Printf("Current version: %d\n", currentVersion)
// Migrate to latest version
var err = db.MigrateToVersion(conn, 3, migrations)
if err != None {
fmt.Printf("Migration error: %v\n", err)
return
}
fmt.Println("Migrated to version 3")
// Rollback to version 1
var downErr = db.MigrateToVersion(conn, 1, migrations)
if downErr != None {
fmt.Printf("Rollback error: %v\n", downErr)
return
}
fmt.Println("Rolled back to version 1")
// View migration history
var history, _ = db.GetMigrationHistory(conn)
fmt.Println("Applied migrations:")
for mig in history {
fmt.Printf(" v%d: %s\n", mig["version"], mig["name"])
}
db.Close(conn)
}
|
File-Based Migrations
Migrations can be stored in external files for better organization and version control.
Method 1: JSON File
Store all migrations in a single JSON file:
migrations.json:
| [
{
"version": 1,
"name": "create_users_table",
"up": "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)",
"down": "DROP TABLE users"
},
{
"version": 2,
"name": "add_users_email",
"up": "ALTER TABLE users ADD COLUMN email TEXT",
"down": "ALTER TABLE users DROP COLUMN email"
}
]
|
Load and apply:
| Load JSON Migrations |
|---|
| import db
import json
import file
function loadMigrations(filepath string) array {
var content, err = file.ReadFile(filepath)
if err != None {
return []
}
var migrations, parseErr = json.Unmarshal(content)
if parseErr != None {
return []
}
return migrations
}
function main() {
var conn, _ = db.Open("sqlite3", "myapp.db")
// Load migrations from JSON file
var migrations = loadMigrations("migrations.json")
// Get current version
var currentVersion, _ = db.GetMigrationVersion(conn)
// Apply pending migrations
for migration in migrations {
if migration["version"] > currentVersion {
db.MigrateUp(conn, migration["version"],
migration["name"], migration["up"])
}
}
db.Close(conn)
}
|
Method 2: Individual SQL Files
Store each migration in a separate SQL file:
migrations/001_create_users.sql:
| -- Migration: Create users table
-- Version: 1
-- Name: create_users_table
-- UP
CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
email TEXT UNIQUE NOT NULL
);
-- DOWN
DROP TABLE users;
|
migrations/002_add_age.sql:
| -- Migration: Add age column
-- Version: 2
-- Name: add_users_age
-- UP
ALTER TABLE users ADD COLUMN age INTEGER;
-- DOWN
ALTER TABLE users DROP COLUMN age;
|
Load and apply:
| Load SQL Migrations |
|---|
| import db
import file
import path
import strings
function parseSQLFile(filepath string) map {
var content, _ = file.Read(filepath)
var lines = strings.Split(content, "\n")
var version = 0
var name = ""
var upSQL = ""
var downSQL = ""
var inUpSection = false
var inDownSection = false
for line in lines {
var trimmed = strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "-- Version:") {
var parts = strings.Split(trimmed, ":")
if len(parts) == 2 {
version = cast.ToInt(strings.TrimSpace(parts[1]))
}
} else if strings.HasPrefix(trimmed, "-- Name:") {
var parts = strings.Split(trimmed, ":")
if len(parts) == 2 {
name = strings.TrimSpace(parts[1])
}
} else if trimmed == "-- UP" {
inUpSection = true
inDownSection = false
} else if trimmed == "-- DOWN" {
inUpSection = false
inDownSection = true
} else if trimmed != "" && !strings.HasPrefix(trimmed, "--") {
if inUpSection {
upSQL = upSQL + " " + trimmed
} else if inDownSection {
downSQL = downSQL + " " + trimmed
}
}
}
return {
"version": version,
"name": name,
"up": strings.TrimSpace(upSQL),
"down": strings.TrimSpace(downSQL)
}
}
function loadMigrationsFromDir(dirpath string) array {
var files, _ = path.Glob(dirpath + "/*.sql")
var migrations = []
for filepath in files {
var migration = parseSQLFile(filepath)
if migration["version"] != 0 {
migrations = arrays.Append(migrations, migration)
}
}
return migrations
}
function main() {
var conn, _ = db.Open("sqlite3", "myapp.db")
// Load all SQL migration files
var migrations = loadMigrationsFromDir("migrations")
// Get current version
var currentVersion, _ = db.GetMigrationVersion(conn)
// Apply pending migrations
for migration in migrations {
if migration["version"] > currentVersion {
db.MigrateUp(conn, migration["version"],
migration["name"], migration["up"])
}
}
db.Close(conn)
}
|
Migration Best Practices
1. Always Define Both Up and Down
| Define Up and Down |
|---|
| var migration = {
"version": 1,
"name": "add_column",
"up": "ALTER TABLE users ADD COLUMN phone TEXT",
"down": "ALTER TABLE users DROP COLUMN phone" // Always reversible
}
|
2. Use Sequential Versioning
| Sequential Versioning |
|---|
| // Good - sequential versions
var migrations = [
{"version": 1, ...},
{"version": 2, ...},
{"version": 3, ...}
]
|
3. Test Migrations in Development
| Test Migrations |
|---|
| // Apply migration
db.MigrateUp(conn, 1, "create_table", createSQL)
// Test that it works
var users, _ = db.Query(conn, "SELECT * FROM users")
// Test rollback
db.MigrateDown(conn, 1, dropSQL)
|
4. Keep Migrations Small and Focused
| Small Migrations |
|---|
| // Good - one change per migration
{"version": 1, "name": "create_users_table", ...}
{"version": 2, "name": "add_users_email", ...}
{"version": 3, "name": "create_posts_table", ...}
|
5. Version History is Automatic
The module automatically tracks: - Which migrations have been applied - When they were applied - Current database version
Complete Example
| Complete Example |
|---|
| import db
import fmt
function main() {
// Open database
var conn, err = db.Open("sqlite3", "myapp.db")
if err != None {
fmt.Printf("Error: %v\n", err)
return
}
// Configure connection pool
db.SetMaxOpenConns(conn, 25)
db.SetMaxIdleConns(conn, 5)
db.SetConnMaxLifetime(conn, 300)
// Create table
var _, createErr = db.Exec(conn, `
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
email TEXT UNIQUE NOT NULL,
age INTEGER
)
`)
if createErr != None {
fmt.Printf("Create table error: %v\n", createErr)
db.Close(conn)
return
}
// Insert data (ORM style)
var id, insertErr = db.Create(conn, "users", {
"name": "Alice",
"email": "alice@example.com",
"age": 30
})
fmt.Printf("Inserted user ID: %d\n", id)
// Query data (SQL style)
var users, queryErr = db.Query(conn, "SELECT * FROM users WHERE age >= ?", 18)
if queryErr == None {
fmt.Println("Users:")
for user in users {
fmt.Printf(" - %s (%d)\n", user["name"], user["age"])
}
}
// Update data (ORM style)
var updateCount, updateErr = db.Update(conn, "users",
{"email": "alice@example.com"},
{"age": 31})
fmt.Printf("Updated %d users\n", updateCount)
// Count records
var total, countErr = db.Count(conn, "users", {})
fmt.Printf("Total users: %d\n", total)
// Close connection
db.Close(conn)
}
|
Integration with Other Modules
JSON Integration
| JSON Integration |
|---|
| import db
import json
var users, _ = db.Query(conn, "SELECT * FROM users")
var jsonStr, _ = json.Marshal(users)
fmt.Println(jsonStr)
// Output: [{"id":1,"name":"Alice","age":30},{"id":2,"name":"Bob","age":25}]
|
Map Integration
| Map Integration |
|---|
| var user, _ = db.FindOne(conn, "users", {"id": 1})
// Direct map access
var name = user["name"]
var age = user["age"]
// Dot notation access
var email = user.email
|
Error Handling
The database module follows Harneet's error tuple pattern:
| Error Handling |
|---|
| var conn, err = db.Open("sqlite3", "test.db")
if err != None {
fmt.Printf("Connection error: %v\n", err)
return
}
var users, queryErr = db.Query(conn, "SELECT * FROM users")
if queryErr != None {
fmt.Printf("Query error: %v\n", queryErr)
return
}
// Safe to use users
for user in users {
fmt.Println(user["name"])
}
|
Best Practices
1. Always Close Connections
| Always Close Connections |
|---|
| var conn, _ = db.Open("sqlite3", "app.db")
// ... use connection
db.Close(conn)
|
2. Use Connection Pooling for Production
| Connection Pooling |
|---|
| var conn, _ = db.Open("postgres", dsn)
db.SetMaxOpenConns(conn, 25)
db.SetMaxIdleConns(conn, 5)
db.SetConnMaxLifetime(conn, 300)
|
3. Use Transactions for Multiple Operations
| Use Transactions |
|---|
| var tx, _ = db.Begin(conn)
// Execute multiple related operations
var _, err1 = db.TxExec(tx, "INSERT ...")
var _, err2 = db.TxExec(tx, "UPDATE ...")
if err1 != None or err2 != None {
db.Rollback(tx)
} else {
db.Commit(tx)
}
|
4. Use Parameterized Queries
| Parameterized Queries |
|---|
| // ✅ Good - prevents SQL injection
var users, _ = db.Query(conn, "SELECT * FROM users WHERE id = ?", userId)
// ❌ Bad - vulnerable to SQL injection
var sql = "SELECT * FROM users WHERE id = " + userId
var users, _ = db.Query(conn, sql)
|
5. Handle Errors Appropriately
| Handle Errors |
|---|
| var user, err = db.FindOne(conn, "users", {"id": 1})
if err != None {
// Log error, return default, or propagate
fmt.Printf("Database error: %v\n", err)
return
}
|
Summary
The db module provides enterprise-grade database functionality:
- ✅ Multiple drivers: SQLite3, PostgreSQL, MySQL
- ✅ Connection pooling: Production-ready connection management
- ✅ SQLC-style queries: Type-safe parameter binding
- ✅ Transaction support: ACID compliance with manual control
- ✅ GORM-inspired ORM: Simple CRUD operations
- ✅ Map-based results: Seamless Harneet integration
- ✅ JSON ready: Direct serialization support
Perfect for building web applications, APIs, and data-driven tools!