2 minute read • February 14th 2023 • Last updated on July 15th 2023

Implementing Audit Log using GORM in Go

•

gORM

In many applications, it's essential to keep track of changes made to data in the database for auditing and debugging purposes. One common approach is to implement an audit log, which records information about the operations performed on the data. In this blog post, we will explore how to implement an audit log using GORM, a popular ORM library for Go.

Prerequisites

Before we dive into the implementation, make sure you have the following prerequisites in place:

Go installed on your machine
GORM library installed (go get -u gorm.io/gorm)
SQLite database installed (go get -u gorm.io/driver/sqlite)

Initial setup

To get started, we need to set up a SQLite database and define the required models. In this example, we'll create a sample table called YourTable to demonstrate the CREATE, UPDATE, and DELETE operations.

To set up the SQLite database and define the models, follow these steps:

Connect to the SQLite database using GORM. Ensure that you import the necessary packages (gorm.io/gorm and gorm.io/driver/sqlite).
A YourTable model struct is defined, which represents your application data. Sample fields are included such as Id, Name, Description, and CreatedAt.
Use db.AutoMigrate(&YourTable{}) to automatically create or migrate the your_tables table based on the defined model.

Here's an example of the sample audit log code:

package main

import (
	"encoding/json"
	"fmt"
	"reflect"
	"time"

	"github.com/oklog/ulid/v2"
	"github.com/sirupsen/logrus"
	"gorm.io/datatypes"
	"gorm.io/driver/sqlite"
	"gorm.io/gorm"
)

// YourTable represents sample table
type YourTable struct {
	Id          string         `json:"id" gorm:"primaryKey"`
	Name        string         `json:"name"`
	Description string         `json:"description"`
	CreatedAt   time.Time      `json:"created_at"`
	UpdatedAt   time.Time      `json:"updated_at"`
	DeletedAt   gorm.DeletedAt `json:"deleted_at"`
}

// AuditLog represents the audit log model
type AuditLog struct {
	Id            string         `json:"id" gorm:"primaryKey"`
	TableName     string         `json:"table_name"`
	OperationType string         `json:"operation_type"`
	ObjectId      string         `json:"object_id"`
	Data          datatypes.JSON `json:"data"`
	UserId        string         `json:"user_id"`
	CreatedAt     time.Time      `json:"created_at"`
}

// Sample method to retrieve user currently using the system
func getCurrentUser() string {
	return "admin"
}

func main() {
	// Connect to the SQLite database
	db, err := gorm.Open(sqlite.Open("your_database.db"), &gorm.Config{})
	if err != nil {
		panic("failed to connect to database")
	}

	db.AutoMigrate(
		&YourTable{},
		&AuditLog{},
	)

	// Register the audit log callbacks and perform migrations
	err = RegisterCallbacks(db)
	if err != nil {
		panic("failed to register audit log callbacks")
	}

	sampleTable := &YourTable{
		Id:          ulid.Make().String(),
		Name:        "John Doe",
		Description: "This is a sample text",
		CreatedAt:   time.Now(),
	}

	if err := db.Create(&sampleTable).Error; err != nil {
		panic(err)
	}

	sampleTable.Description = "Changed description"

	if err := db.Save(&sampleTable).Error; err != nil {
		panic(err)
	}

	if err := db.Delete(&sampleTable).Error; err != nil {
		panic(err)
	}

	auditLogs := []*AuditLog{}

	if err := db.Find(&auditLogs).Error; err != nil {
		panic(err)
	}

    // Print the audit logs to view them. You should see a CREATE, UPDATE and DELETE log.

}

func getKeyFromData(key string, data map[string]interface{}) string {
	objId, ok := data[key]
	if !ok {
		return ""
	}
	return objId.(string)
}

func prepareData(data map[string]interface{}) datatypes.JSON {
	dataByte, _ := json.Marshal(&data)
	return dataByte
}

func createAuditLog(db *gorm.DB) {
	if db.Statement.Schema != nil && db.Statement.Schema.Table == "audit_logs" || db.Error != nil {
		return
	}

	recordMap, err := getDataBeforeOperation(db)
	if err != nil {
		return
	}
	objId := getKeyFromData("id", recordMap)
	auditLog := &AuditLog{
		Id:            ulid.Make().String(),
		TableName:     db.Statement.Schema.Table,
		OperationType: "CREATE",
		ObjectId:      objId,
		Data:          prepareData(recordMap),
		UserId:        getCurrentUser(),
	}
	if err := db.Session(&gorm.Session{SkipHooks: true, NewDB: true}).Table("audit_logs").Create(auditLog).Error; err != nil {
		logrus.Error(fmt.Errorf("error in audit log creation: %s", err.Error()))
		return
	}
}

func updateAuditLog(db *gorm.DB) {
	if db.Statement.Schema != nil && db.Statement.Schema.Table == "audit_logs" || db.Error != nil {
		return
	}

	recordMap, err := getDataBeforeOperation(db)
	if err != nil {
		return
	}
	objId := getKeyFromData("id", recordMap)
	auditLog := &AuditLog{
		Id:            ulid.Make().String(),
		TableName:     db.Statement.Schema.Table,
		OperationType: "UPDATE",
		ObjectId:      objId,
		Data:          prepareData(recordMap),
		UserId:        getCurrentUser(),
	}
	if err := db.Session(&gorm.Session{SkipHooks: true, NewDB: true}).Table("audit_logs").Create(auditLog).Error; err != nil {
		logrus.Error(fmt.Errorf("error in audit log creation: %s", err.Error()))
		return
	}
}

func deleteAuditLog(db *gorm.DB) {
	if db.Statement.Schema != nil && db.Statement.Schema.Table == "audit_logs" || db.Error != nil {
		return
	}

	recordMap, err := getDataBeforeOperation(db)
	if err != nil {
		return
	}
	objId := getKeyFromData("id", recordMap)
	auditLog := &AuditLog{
		Id:            ulid.Make().String(),
		TableName:     db.Statement.Schema.Table,
		OperationType: "DELETE",
		ObjectId:      objId,
		Data:          prepareData(recordMap),
		UserId:        getCurrentUser(),
	}
	if err := db.Session(&gorm.Session{SkipHooks: true, NewDB: true}).Table("audit_logs").Create(auditLog).Error; err != nil {
		logrus.Error(fmt.Errorf("error in audit log creation: %s", err.Error()))
		return
	}
}

func getDataBeforeOperation(db *gorm.DB) (map[string]interface{}, error) {
	objMap := map[string]interface{}{}
	if db.Error == nil && !db.DryRun {
		objectType := reflect.TypeOf(db.Statement.ReflectValue.Interface())

		// Create a new instance of the object type
		targetObj := reflect.New(objectType).Interface()

		primaryKeyValue := ""
		value := db.Statement.ReflectValue

		// Check if the value is a struct
		if value.Kind() == reflect.Struct {
			primaryKeyValue = value.FieldByName("Id").String()
		}

		// Fetch the target object separately
		if err := db.Session(&gorm.Session{SkipHooks: true, NewDB: true}).Where("id = ?", primaryKeyValue).First(&targetObj).Error; err != nil {
			logrus.Error(fmt.Errorf("gorm callback: error while finding target object: %s", err.Error()))
			return nil, err
		}

		jsonBytes, err := json.Marshal(targetObj)
		if err != nil {
			logrus.Error(fmt.Errorf("gorm callback: error while marshalling json data: %s", err.Error()))
			return nil, err
		}
		json.Unmarshal(jsonBytes, &objMap)
	}
	return objMap, nil
}

func RegisterCallbacks(db *gorm.DB) error {
	db.Callback().Create().After("gorm:create").Register("custom_plugin:create_audit_log", createAuditLog)
	db.Callback().Update().After("gorm:update").Register("custom_plugin:update_audit_log", updateAuditLog)
	db.Callback().Delete().Before("gorm:delete").Register("custom_plugin:delete_audit_log", deleteAuditLog)
	return nil
}

The changes are currently saved in your_database.db SQLite database file.

With the database and model set up, we can now proceed to implement the audit log functionality using GORM.

Implementing the Audit Log

In the above code, we have defined several functions that are responsible for creating audit log entries. These functions are registered as GORM callbacks for the respective operations (create, update, and delete).

The createAuditLog, updateAuditLog, and deleteAuditLog functions retrieve the necessary data before the operation using the getDataBeforeOperation function. They construct an AuditLog object with the required information and use GORM's Create method to store the audit log entry in the audit_logs table. The callbacks ensure that creating an audit log does not trigger that callback again by performing the check at the start of each callback.

The getDataBeforeOperation function retrieves the data before the database operation by reflecting on the GORM statement's reflected value. It is essential to note that a new GORM session should be initialized with a NewDB so that the existing conditions(of the create, update or delete query) do not affect the find query. It creates a new instance of the object type, fetches the target object from the database, and serializes it to JSON. Finally, it deserializes the JSON into a map.

The RegisterCallbacks function is responsible for registering the audit log callbacks for the respective GORM operations. It also ensures that the necessary database table (your_tables) is created or migrated using db.AutoMigrate.

Conclusion

Implementing an audit log using GORM allows you to track changes made to your application data, enabling auditing, debugging, and maintaining data integrity. By following the steps outlined in this blog post, you can easily incorporate audit log functionality into your Go applications. Remember to customize the code to fit your specific application needs, such as the structure of your application data and any additional fields you want to include in the audit log entries. Having an audit log in place provides transparency, accountability, and traceability for data changes, which is essential for many applications.