Skip to content

Database Module

The db module provides comprehensive SQL database connectivity with connection pooling, transaction support, and GORM-inspired ORM features. It supports SQLite3, PostgreSQL, and MySQL databases with SQLC-like query workflows.

Features

  • Multiple Database Support: SQLite3, PostgreSQL, MySQL
  • Connection Pooling: Configurable pool size and connection lifetime
  • SQLC-style Queries: Type-safe query execution with parameter binding
  • Transaction Support: Manual and automatic transaction management
  • GORM-inspired ORM: Simple CRUD operations with map-based data
  • Map Integration: Results returned as Harneet maps
  • JSON Ready: Seamless JSON serialization of query results

Import

Import
import db

Connection Management

Open(driver string, dsn string) (connection, error)

Opens a database connection with the specified driver and data source name.

Supported Drivers: - sqlite3 - SQLite file-based database - postgres - PostgreSQL database - mysql - MySQL database

Example:

Open Connection Example
import db
import fmt

// SQLite3
var conn, err = db.Open("sqlite3", "myapp.db")

// PostgreSQL
var conn, err = db.Open("postgres", "host=localhost user=postgres password=secret dbname=mydb sslmode=disable")

// MySQL
var conn, err = db.Open("mysql", "user:password@tcp(localhost:3306)/dbname")

if err != None {
    fmt.Printf("Connection failed: %v\n", err)
    return
}

Close(conn connection) error

Closes the database connection and releases resources.

Close Connection
1
2
3
4
var err = db.Close(conn)
if err != None {
    fmt.Printf("Close error: %v\n", err)
}

Ping(conn connection) error

Verifies the database connection is alive.

Ping Connection
1
2
3
4
var err = db.Ping(conn)
if err != None {
    fmt.Println("Database connection lost")
}

Connection Pool Configuration

SetMaxOpenConns(conn connection, n int) None

Sets the maximum number of open connections to the database.

SetMaxOpenConns
db.SetMaxOpenConns(conn, 25)  // Max 25 connections

SetMaxIdleConns(conn connection, n int) None

Sets the maximum number of idle connections in the pool.

SetMaxIdleConns
db.SetMaxIdleConns(conn, 5)  // Keep 5 idle connections

SetConnMaxLifetime(conn connection, seconds int) None

Sets the maximum lifetime of a connection in seconds.

SetConnMaxLifetime
db.SetConnMaxLifetime(conn, 300)  // 5 minutes max lifetime

Query Execution (SQLC-style)

Query(conn connection, sql string, ...args) (array[map], error)

Executes a SQL query that returns multiple rows. Results are returned as an array of maps where each map represents a row.

Query Example
// Query with no parameters
var users, err = db.Query(conn, "SELECT * FROM users")

// Query with parameters (uses ? placeholders)
var adults, err = db.Query(conn, "SELECT * FROM users WHERE age >= ?", 18)

// Multiple parameters
var results, err = db.Query(conn, 
    "SELECT * FROM users WHERE age >= ? AND city = ?", 
    18, "New York")

if err == None {
    for row in results {
        fmt.Printf("ID: %d, Name: %s, Age: %d\n", 
            row["id"], row["name"], row["age"])
    }
}

QueryRow(conn connection, sql string, ...args) (map, error)

Executes a SQL query that returns a single row.

QueryRow Example
1
2
3
4
5
6
7
var user, err = db.QueryRow(conn, "SELECT * FROM users WHERE id = ?", 1)
if err != None {
    fmt.Println("User not found")
    return
}

fmt.Printf("User: %s, Email: %s\n", user["name"], user["email"])

Exec(conn connection, sql string, ...args) (result, error)

Executes a SQL statement (INSERT, UPDATE, DELETE) that doesn't return rows.

Exec Example
// INSERT
var result, err = db.Exec(conn, 
    "INSERT INTO users (name, email, age) VALUES (?, ?, ?)",
    "Alice", "alice@example.com", 30)

// UPDATE
var result, err = db.Exec(conn,
    "UPDATE users SET age = ? WHERE id = ?",
    31, 1)

// DELETE
var result, err = db.Exec(conn,
    "DELETE FROM users WHERE age < ?",
    18)

LastInsertId(result) (int, error)

Returns the last inserted ID from an INSERT operation.

LastInsertId Example
1
2
3
4
5
var result, err = db.Exec(conn, "INSERT INTO users (name) VALUES (?)", "Bob")
if err == None {
    var id, _ = db.LastInsertId(result)
    fmt.Printf("New user ID: %d\n", id)
}

RowsAffected(result) (int, error)

Returns the number of rows affected by an UPDATE or DELETE operation.

RowsAffected Example
1
2
3
4
5
var result, err = db.Exec(conn, "UPDATE users SET active = 1")
if err == None {
    var count, _ = db.RowsAffected(result)
    fmt.Printf("Updated %d users\n", count)
}

Transaction Support

Begin(conn connection) (transaction, error)

Starts a new database transaction.

Begin Transaction
1
2
3
4
5
var tx, err = db.Begin(conn)
if err != None {
    fmt.Println("Failed to start transaction")
    return
}

Commit(tx transaction) error

Commits the current transaction.

Commit Transaction
1
2
3
4
var err = db.Commit(tx)
if err != None {
    fmt.Println("Commit failed")
}

Rollback(tx transaction) error

Rolls back the current transaction.

Rollback Transaction
1
2
3
4
var err = db.Rollback(tx)
if err != None {
    fmt.Println("Rollback failed")
}

TxQuery(tx transaction, sql string, ...args) (array[map], error)

Executes a query within a transaction.

TxQuery Example
var tx, _ = db.Begin(conn)
var users, err = db.TxQuery(tx, "SELECT * FROM users WHERE age > ?", 18)

TxExec(tx transaction, sql string, ...args) (result, error)

Executes a statement within a transaction.

TxExec Example
var tx, _ = db.Begin(conn)
var result, err = db.TxExec(tx, "INSERT INTO users (name) VALUES (?)", "Charlie")

Transaction Example

Transaction Example
import db
import fmt

function transferMoney(conn any, fromId int, toId int, amount int) error {
    // Start transaction
    var tx, err = db.Begin(conn)
    if err != None {
        return err
    }

    // Deduct from sender
    var result1, err1 = db.TxExec(tx, 
        "UPDATE accounts SET balance = balance - ? WHERE user_id = ?",
        amount, fromId)

    // Add to receiver
    var result2, err2 = db.TxExec(tx,
        "UPDATE accounts SET balance = balance + ? WHERE user_id = ?",
        amount, toId)

    // Check if both succeeded
    if err1 != None or err2 != None {
        db.Rollback(tx)
        fmt.Println("Transaction rolled back")
        return err1
    }

    // Commit transaction
    var commitErr = db.Commit(tx)
    if commitErr != None {
        db.Rollback(tx)
        return commitErr
    }

    fmt.Println("Transfer successful")
    return None
}

ORM Features (GORM-inspired)

Create(conn connection, table string, data map) (int, error)

Inserts a new record into the database. Returns the inserted ID.

Insert Example
var userData = {
    "name": "Alice",
    "email": "alice@example.com",
    "age": 30
}

var id, err = db.Create(conn, "users", userData)
if err == None {
    fmt.Printf("Created user with ID: %d\n", id)
}

Find(conn connection, table string, conditions map) (array[map], error)

Retrieves records from the database matching the conditions.

Find Example
// Find all users aged 30
var users, err = db.Find(conn, "users", {"age": 30})

// Find users in a specific city
var users, err = db.Find(conn, "users", {"city": "New York"})

// Find with multiple conditions (AND)
var users, err = db.Find(conn, "users", {"age": 30, "city": "New York"})

if err == None {
    for user in users {
        fmt.Printf("User: %s\n", user["name"])
    }
}

FindOne(conn connection, table string, conditions map) (map, error)

Retrieves a single record from the database.

FindOne Example
1
2
3
4
5
6
7
var user, err = db.FindOne(conn, "users", {"id": 1})
if err != None {
    fmt.Println("User not found")
    return
}

fmt.Printf("Found: %s\n", user["name"])

Update(conn connection, table string, conditions map, data map) (int, error)

Updates records in the database. Returns the number of rows affected.

Update Example
// Update specific user
var count, err = db.Update(conn, "users", 
    {"id": 1},
    {"age": 31, "city": "Boston"})

// Update multiple users
var count, err = db.Update(conn, "users",
    {"city": "New York"},
    {"timezone": "EST"})

if err == None {
    fmt.Printf("Updated %d records\n", count)
}

Delete(conn connection, table string, conditions map) (int, error)

Deletes records from the database. Returns the number of rows deleted.

Safety: Conditions map is required to prevent accidental deletion of all records.

Delete Example
1
2
3
4
5
6
7
8
9
// Delete specific user
var count, err = db.Delete(conn, "users", {"id": 1})

// Delete multiple users
var count, err = db.Delete(conn, "users", {"age": 0})

if err == None {
    fmt.Printf("Deleted %d records\n", count)
}

Count(conn connection, table string, conditions map) (int, error)

Counts records in the database matching the conditions.

Count Example
1
2
3
4
5
6
7
8
9
// Count all users
var total, err = db.Count(conn, "users", {})

// Count users with conditions
var adults, err = db.Count(conn, "users", {"age": 18})

if err == None {
    fmt.Printf("Found %d users\n", total)
}

Database Migrations

The database module includes comprehensive migration support with both up (apply) and down (rollback) capabilities, similar to Rails migrations or Flyway.

Migration Functions

MigrateUp(conn connection, version int, name string, upSQL string) error

Applies a migration to the database. Migrations are tracked in a schema_migrations table.

MigrateUp Example
var err = db.MigrateUp(conn, 1, "create_users_table",
    "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT, email TEXT)")

MigrateDown(conn connection, version int, downSQL string) error

Rolls back a migration. Only applies if the migration was previously applied.

MigrateDown Example
var err = db.MigrateDown(conn, 1, "DROP TABLE users")

GetMigrationVersion(conn connection) (int, error)

Returns the current migration version number.

GetMigrationVersion Example
var version, err = db.GetMigrationVersion(conn)
fmt.Printf("Current version: %d\n", version)

GetMigrationHistory(conn connection) (array[map], error)

Returns all applied migrations with version, name, and timestamp.

GetMigrationHistory Example
1
2
3
4
5
6
7
var history, err = db.GetMigrationHistory(conn)
for migration in history {
    fmt.Printf("v%d: %s (applied at: %d)\n", 
        migration["version"], 
        migration["name"],
        migration["applied_at"])
}

MigrateToVersion(conn connection, targetVersion int, migrations array) error

Migrates to a specific version (up or down automatically). Migrations array should contain migration maps.

MigrateTo Example
var migrations = [
    {
        "version": 1,
        "name": "create_users",
        "up": "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)",
        "down": "DROP TABLE users"
    },
    {
        "version": 2,
        "name": "add_users_email",
        "up": "ALTER TABLE users ADD COLUMN email TEXT",
        "down": "ALTER TABLE users DROP COLUMN email"
    }
]

// Migrate to version 2
var err = db.MigrateToVersion(conn, 2, migrations)

// Rollback to version 1
var err = db.MigrateToVersion(conn, 1, migrations)

Migration Example

Migration Example
import db
import fmt

function main() {
    var conn, _ = db.Open("sqlite3", "myapp.db")

    // Define migrations
    var migrations = [
        {
            "version": 1,
            "name": "create_users_table",
            "up": "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL, email TEXT UNIQUE)",
            "down": "DROP TABLE users"
        },
        {
            "version": 2,
            "name": "add_users_age",
            "up": "ALTER TABLE users ADD COLUMN age INTEGER",
            "down": "ALTER TABLE users DROP COLUMN age"
        },
        {
            "version": 3,
            "name": "create_posts_table",
            "up": "CREATE TABLE posts (id INTEGER PRIMARY KEY, user_id INTEGER, title TEXT, content TEXT, FOREIGN KEY(user_id) REFERENCES users(id))",
            "down": "DROP TABLE posts"
        }
    ]

    // Check current version
    var currentVersion, _ = db.GetMigrationVersion(conn)
    fmt.Printf("Current version: %d\n", currentVersion)

    // Migrate to latest version
    var err = db.MigrateToVersion(conn, 3, migrations)
    if err != None {
        fmt.Printf("Migration error: %v\n", err)
        return
    }

    fmt.Println("Migrated to version 3")

    // Rollback to version 1
    var downErr = db.MigrateToVersion(conn, 1, migrations)
    if downErr != None {
        fmt.Printf("Rollback error: %v\n", downErr)
        return
    }

    fmt.Println("Rolled back to version 1")

    // View migration history
    var history, _ = db.GetMigrationHistory(conn)
    fmt.Println("Applied migrations:")
    for mig in history {
        fmt.Printf("  v%d: %s\n", mig["version"], mig["name"])
    }

    db.Close(conn)
}

File-Based Migrations

Migrations can be stored in external files for better organization and version control.

Method 1: JSON File

Store all migrations in a single JSON file:

migrations.json:

[
  {
    "version": 1,
    "name": "create_users_table",
    "up": "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)",
    "down": "DROP TABLE users"
  },
  {
    "version": 2,
    "name": "add_users_email",
    "up": "ALTER TABLE users ADD COLUMN email TEXT",
    "down": "ALTER TABLE users DROP COLUMN email"
  }
]

Load and apply:

Load JSON Migrations
import db
import json
import file

function loadMigrations(filepath string) array {
    var content, err = file.ReadFile(filepath)
    if err != None {
        return []
    }

    var migrations, parseErr = json.Unmarshal(content)
    if parseErr != None {
        return []
    }

    return migrations
}

function main() {
    var conn, _ = db.Open("sqlite3", "myapp.db")

    // Load migrations from JSON file
    var migrations = loadMigrations("migrations.json")

    // Get current version
    var currentVersion, _ = db.GetMigrationVersion(conn)

    // Apply pending migrations
    for migration in migrations {
        if migration["version"] > currentVersion {
            db.MigrateUp(conn, migration["version"], 
                         migration["name"], migration["up"])
        }
    }

    db.Close(conn)
}

Method 2: Individual SQL Files

Store each migration in a separate SQL file:

migrations/001_create_users.sql:

-- Migration: Create users table
-- Version: 1
-- Name: create_users_table

-- UP
CREATE TABLE users (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,
    email TEXT UNIQUE NOT NULL
);

-- DOWN
DROP TABLE users;

migrations/002_add_age.sql:

1
2
3
4
5
6
7
8
9
-- Migration: Add age column
-- Version: 2
-- Name: add_users_age

-- UP
ALTER TABLE users ADD COLUMN age INTEGER;

-- DOWN
ALTER TABLE users DROP COLUMN age;

Load and apply:

Load SQL Migrations
import db
import file
import path
import strings

function parseSQLFile(filepath string) map {
    var content, _ = file.Read(filepath)
    var lines = strings.Split(content, "\n")

    var version = 0
    var name = ""
    var upSQL = ""
    var downSQL = ""
    var inUpSection = false
    var inDownSection = false

    for line in lines {
        var trimmed = strings.TrimSpace(line)

        if strings.HasPrefix(trimmed, "-- Version:") {
            var parts = strings.Split(trimmed, ":")
            if len(parts) == 2 {
                version = cast.ToInt(strings.TrimSpace(parts[1]))
            }
        } else if strings.HasPrefix(trimmed, "-- Name:") {
            var parts = strings.Split(trimmed, ":")
            if len(parts) == 2 {
                name = strings.TrimSpace(parts[1])
            }
        } else if trimmed == "-- UP" {
            inUpSection = true
            inDownSection = false
        } else if trimmed == "-- DOWN" {
            inUpSection = false
            inDownSection = true
        } else if trimmed != "" && !strings.HasPrefix(trimmed, "--") {
            if inUpSection {
                upSQL = upSQL + " " + trimmed
            } else if inDownSection {
                downSQL = downSQL + " " + trimmed
            }
        }
    }

    return {
        "version": version,
        "name": name,
        "up": strings.TrimSpace(upSQL),
        "down": strings.TrimSpace(downSQL)
    }
}

function loadMigrationsFromDir(dirpath string) array {
    var files, _ = path.Glob(dirpath + "/*.sql")
    var migrations = []

    for filepath in files {
        var migration = parseSQLFile(filepath)
        if migration["version"] != 0 {
            migrations = arrays.Append(migrations, migration)
        }
    }

    return migrations
}

function main() {
    var conn, _ = db.Open("sqlite3", "myapp.db")

    // Load all SQL migration files
    var migrations = loadMigrationsFromDir("migrations")

    // Get current version
    var currentVersion, _ = db.GetMigrationVersion(conn)

    // Apply pending migrations
    for migration in migrations {
        if migration["version"] > currentVersion {
            db.MigrateUp(conn, migration["version"],
                         migration["name"], migration["up"])
        }
    }

    db.Close(conn)
}

Migration Best Practices

1. Always Define Both Up and Down

Define Up and Down
1
2
3
4
5
6
var migration = {
    "version": 1,
    "name": "add_column",
    "up": "ALTER TABLE users ADD COLUMN phone TEXT",
    "down": "ALTER TABLE users DROP COLUMN phone"  // Always reversible
}

2. Use Sequential Versioning

Sequential Versioning
1
2
3
4
5
6
// Good - sequential versions
var migrations = [
    {"version": 1, ...},
    {"version": 2, ...},
    {"version": 3, ...}
]

3. Test Migrations in Development

Test Migrations
1
2
3
4
5
6
7
8
// Apply migration
db.MigrateUp(conn, 1, "create_table", createSQL)

// Test that it works
var users, _ = db.Query(conn, "SELECT * FROM users")

// Test rollback
db.MigrateDown(conn, 1, dropSQL)

4. Keep Migrations Small and Focused

Small Migrations
1
2
3
4
// Good - one change per migration
{"version": 1, "name": "create_users_table", ...}
{"version": 2, "name": "add_users_email", ...}
{"version": 3, "name": "create_posts_table", ...}

5. Version History is Automatic

The module automatically tracks: - Which migrations have been applied - When they were applied - Current database version

Complete Example

Complete Example
import db
import fmt

function main() {
    // Open database
    var conn, err = db.Open("sqlite3", "myapp.db")
    if err != None {
        fmt.Printf("Error: %v\n", err)
        return
    }

    // Configure connection pool
    db.SetMaxOpenConns(conn, 25)
    db.SetMaxIdleConns(conn, 5)
    db.SetConnMaxLifetime(conn, 300)

    // Create table
    var _, createErr = db.Exec(conn, `
        CREATE TABLE IF NOT EXISTS users (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            name TEXT NOT NULL,
            email TEXT UNIQUE NOT NULL,
            age INTEGER
        )
    `)
    if createErr != None {
        fmt.Printf("Create table error: %v\n", createErr)
        db.Close(conn)
        return
    }

    // Insert data (ORM style)
    var id, insertErr = db.Create(conn, "users", {
        "name": "Alice",
        "email": "alice@example.com",
        "age": 30
    })
    fmt.Printf("Inserted user ID: %d\n", id)

    // Query data (SQL style)
    var users, queryErr = db.Query(conn, "SELECT * FROM users WHERE age >= ?", 18)
    if queryErr == None {
        fmt.Println("Users:")
        for user in users {
            fmt.Printf("  - %s (%d)\n", user["name"], user["age"])
        }
    }

    // Update data (ORM style)
    var updateCount, updateErr = db.Update(conn, "users",
        {"email": "alice@example.com"},
        {"age": 31})
    fmt.Printf("Updated %d users\n", updateCount)

    // Count records
    var total, countErr = db.Count(conn, "users", {})
    fmt.Printf("Total users: %d\n", total)

    // Close connection
    db.Close(conn)
}

Integration with Other Modules

JSON Integration

JSON Integration
1
2
3
4
5
6
7
import db
import json

var users, _ = db.Query(conn, "SELECT * FROM users")
var jsonStr, _ = json.Marshal(users)
fmt.Println(jsonStr)
// Output: [{"id":1,"name":"Alice","age":30},{"id":2,"name":"Bob","age":25}]

Map Integration

Map Integration
1
2
3
4
5
6
7
8
var user, _ = db.FindOne(conn, "users", {"id": 1})

// Direct map access
var name = user["name"]
var age = user["age"]

// Dot notation access
var email = user.email

Error Handling

The database module follows Harneet's error tuple pattern:

Error Handling
var conn, err = db.Open("sqlite3", "test.db")
if err != None {
    fmt.Printf("Connection error: %v\n", err)
    return
}

var users, queryErr = db.Query(conn, "SELECT * FROM users")
if queryErr != None {
    fmt.Printf("Query error: %v\n", queryErr)
    return
}

// Safe to use users
for user in users {
    fmt.Println(user["name"])
}

Best Practices

1. Always Close Connections

Always Close Connections
1
2
3
var conn, _ = db.Open("sqlite3", "app.db")
// ... use connection
db.Close(conn)

2. Use Connection Pooling for Production

Connection Pooling
1
2
3
4
var conn, _ = db.Open("postgres", dsn)
db.SetMaxOpenConns(conn, 25)
db.SetMaxIdleConns(conn, 5)
db.SetConnMaxLifetime(conn, 300)

3. Use Transactions for Multiple Operations

Use Transactions
var tx, _ = db.Begin(conn)
// Execute multiple related operations
var _, err1 = db.TxExec(tx, "INSERT ...")
var _, err2 = db.TxExec(tx, "UPDATE ...")

if err1 != None or err2 != None {
    db.Rollback(tx)
} else {
    db.Commit(tx)
}

4. Use Parameterized Queries

Parameterized Queries
1
2
3
4
5
6
// ✅ Good - prevents SQL injection
var users, _ = db.Query(conn, "SELECT * FROM users WHERE id = ?", userId)

// ❌ Bad - vulnerable to SQL injection
var sql = "SELECT * FROM users WHERE id = " + userId
var users, _ = db.Query(conn, sql)

5. Handle Errors Appropriately

Handle Errors
1
2
3
4
5
6
var user, err = db.FindOne(conn, "users", {"id": 1})
if err != None {
    // Log error, return default, or propagate
    fmt.Printf("Database error: %v\n", err)
    return
}

Summary

The db module provides enterprise-grade database functionality:

  • Multiple drivers: SQLite3, PostgreSQL, MySQL
  • Connection pooling: Production-ready connection management
  • SQLC-style queries: Type-safe parameter binding
  • Transaction support: ACID compliance with manual control
  • GORM-inspired ORM: Simple CRUD operations
  • Map-based results: Seamless Harneet integration
  • JSON ready: Direct serialization support

Perfect for building web applications, APIs, and data-driven tools!