sqltrack package

class sqltrack.Client(config: dict | str | Path | None = None, **kwargs)

Bases: object

Creates and manages database connections (currently psycopg.Connection or sqlite3.Connection).

For simple queries, use the execute() method:

client = Client(...)
client.execute(...)

Use a client as context manager to obtain a database connection object:

client = Client(...)
with client as conn
    with conn.cursor() as cursor:
        ...

If all you need is a cursor, you can obtain one directly from a client with the cursor() method:

client = Client(...)
with client.cursor() as cursor:
    ...

Connection parameters are given as kwargs. Use the engine parameter to "postgres" (default) or sqlite. For SQLite, only dbpath is relevant. It defaults to sqltrack.db. For Postgres, common options are:

• user: defaults to USER env var

• dbname: defaults to user

• host

• schema: a shorthand for setting the search_path option.

For the full list of available parameters, see: https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS

Parameters passed from Python take priority, but they may also be passed as environment variables SQLTRACK_DSN_<PARAM> (e.g., SQLTRACK_DSN_USER), or loaded from a config file, by default ./sqltrack.conf.

SQLTrack classes and functions will connect to the database as required. Nested contexts (with blocks) reuse the same connection (reentrant), so they can be used to avoid connecting to the database multiple times over a short period. The connection is closed and any uncomitted changes are comitted when the outermost with block ends. E.g., the following snippet will connect only once:

def do_queries(client, ...):
    with client.cursor() as cursor:
        cursor.execute(...)
        ...

client = Client(...)
with client:
    do_queries(client, ...)
    do_queries(client, ...)
    do_queries(client, ...)

One caveat of this approach is that all changes in a stack of contexts implicitly happen within the same transaction. All pending changes will be rolled back if an exception is raised. You can avoid this by periodically calling commit().

Parameters:

• config – Config dict or path to config file, defaults to SQLTRACK_CONFIG_PATH environment variable, and finally ./sqltrack.conf

• kwargs – Connection parameters

commit()

Convenience function to call commit on the DB connection. Raises RuntimeError when not connected.

cursor() → DBAPICursor

Connect to the DB and return a cursor. Use in with statement:

with client.cursor() as cursor:
    ... cursor things ...

The connection is closed and any changes comitted when the with block ends.

execute(sql, parameters=())

Convenience function that connects to the DB, if necessary, and executes the given query with optional parameters.

executemany(sql, seq_of_parameters)

Convenience function that connects to the DB, if necessary, and calls executemany with the given sequence of parameters.

executescript(sql_script)

Convenience function that connects to the DB, if necessary, and executes the given script.

rollback()

Convenience function to call rollback on the DB connection. Raises RuntimeError when not connected.

class sqltrack.Experiment(client: Client, experiment_id: int | None = None, name: str | None = None)

Bases: object

Helper class to create experiments, as well as runs for experiments.

Note

All methods (except get_run) return self to allow chaining calls, e.g., exp = Experiment(client, id).set_comment("nice").add_tags("tag")

Parameters:

• client – Client object to use.

• experiment_id – ID of the experiment. May be None if name is given.

• name – Name of the experiment. may be None if experiment_id is given.

add_link(kind: str, to_id: int) → Experiment

Add a link to another experiment.

add_tags(*tags: str) → Experiment

Add tags to the experiment.

get_run(run_id: int | str | None = None) → Run

Get a Run object. See its documentation for more details.

remove_link(kind: str, to_id: int) → Experiment

Remove a link to another experiment.

remove_tags(*tags: str) → Experiment

Remove tags from the experiment.

set_comment(comment: str) → Experiment

Set the experiment comment.

set_name(name: str) → Experiment

Set the experiment name.

set_tags(*tags: str) → Experiment

Set tags of the experiment.

class sqltrack.Run(client: Client, experiment_id: int | None = None, run_id: int | str | None = None)

Bases: object

Helper class to manage runs.

Note

If run_id is None, a new run with an unused ID is created.

Note

All methods return self to allow chaining calls, e.g., run = Run(client, exp_id).set_comment("nice").add_tags("tag")

Parameters:

• client – Client object to use.

• experiment_id – ID of the experiment. May be None if an existing run_id is given.

• run_id – ID of the run. If None an unused ID is chosen. If string then load ID from that environment variable.

add_args(args: 'auto' | dict | None = 'auto') → Run

Add parameters to the run. See run_add_args() for details.

add_env(env: 'auto' | dict | None = 'auto') → Run

Add environment variables to the run. See run_add_env() for details.

add_extras(extras: dict | None = None) → Run

Add extras to the run. See run_add_extras() for details.

add_link(kind: str, to_id: int) → Run

Add a link to another run.

add_metrics(metrics: dict, step: int = 0, progress: float = 0.0, updated: 'auto' | datetime | None = 'auto') → Run

Add metrics to the run. See run_add_metrics() for details.

add_tags(*tags: str) → Run

Add tags to the run. See run_add_tags() for details.

remove_args(args: 'auto' | dict | None = 'auto') → Run

Remove parameters from the run. See run_remove_args() for details.

remove_env(env: 'auto' | dict | None = 'auto') → Run

Remove environment variables from the run. See run_remove_env() for details.

remove_extras(*tags: str) → Run

Remove extras from the run. See run_remove_extras() for details.

remove_link(kind: str, to_id: int) → Run

Remove a link to another run.

remove_tags(*tags: str) → Run

Remove tags from the run. See run_remove_tags() for details.

set_args(args: 'auto' | dict | None = 'auto') → Run

Set the run parameters. See run_set_args() for details.

set_comment(comment: str) → Run

Set the run comment.

set_created(dt: 'auto' | datetime | None = 'auto') → Run

Set time_created for the run. See run_set_created() for details.

set_env(env: 'auto' | dict | None = 'auto') → Run

Set the run environment. See run_set_env() for details.

set_extras(*tags: str) → Run

Set the run extras. See run_set_extras() for details.

set_started(dt: 'auto' | datetime | None = 'auto') → Run

Set time_started for the run. See run_set_started() for details.

set_status(status: str) → Run

Set the run status. See run_set_status() for details.

set_tags(*tags: str) → Run

Set the run tags. See run_set_tags() for details.

set_updated(dt: 'auto' | datetime | None = 'auto') → Run

Set time_updated for the run. See run_set_updated() for details.

start(terminated='CANCELLED', started: 'auto' | datetime | None = 'auto', updated: 'auto' | datetime | None = 'auto') → Run

Start the run, setting its status to RUNNING, among other values like time_started, depending on parameters.

Parameters:

• terminated – status in case SIGTERM is received during the run; see also sqltrack.sigterm

• started – what to do about time_started; See run_set_started() for details

• updated – what to do about time_updated; See run_set_updated() for details

stop(status='COMPLETED', updated: 'auto' | datetime | None = 'auto') → Run

Stop the run, setting its status to COMPLETED and time_started to now (default), depending on parameters.

Parameters:

• status – status to set (default: COMPLETED); See run_set_status() for details

• updated – what to do about time_updated; See run_set_updated() for details

track(normal='COMPLETED', exception='FAILED', interrupt='CANCELLED', terminated='CANCELLED', started: 'auto' | datetime | None = 'auto', updated: 'auto' | datetime | None = 'auto') → Run

A context manager to track the execution of the run. This is equivalent to calling start and stop separately with the appropriate status value.

Parameters:

• normal – status in case the run completes normally

• exception – status in case an exception occurs

• interrupt – status in case SIGINT is received during the run

• terminated – status in case SIGTERM is received during the run; see also sqltrack.sigterm

• started – what to do about time_started; See run_set_started() for details

• updated – what to do about time_updated; See run_set_started() for details

sqltrack.experiment_add_link(client: Client, from_id: int, kind: str, to_id: int)

Add a link between two experiments.

sqltrack.experiment_add_tags(client: Client, experiment_id: int, *tags: str)

Add tags to an experiment.

sqltrack.experiment_remove_link(client: Client, from_id: int, kind: str, to_id: int)

Remove a link between two experiments.

sqltrack.experiment_remove_tags(client: Client, experiment_id: int, *tags: str)

Remove tags from an experiment.

sqltrack.experiment_set_comment(client: Client, experiment_id: int, comment: str | None)

Set an experiment comment.

sqltrack.experiment_set_name(client: Client, experiment_id: int, name: str)

Set an experiment name.

sqltrack.experiment_set_tags(client: Client, experiment_id: int, *tags: str)

Set tags of an experiment.

sqltrack.run_add_args(client: Client, run_id: int, args: 'auto' | dict | None = 'auto')

Add some arguments to an existing run. See sqltrack.args.detect_args() for details on detection in auto mode.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• args – the arguments, may be 'auto' (detect arguments), dict, or None (do nothing)

sqltrack.run_add_env(client: Client, run_id: int, env: 'auto' | dict | None = 'auto')

Add some environment variables to an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• env – the environment, may be 'auto' (set to os.environ), dict, or None

sqltrack.run_add_extras(client: Client, run_id: int, extras: dict | None = None)

Add some extras to an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• extras – the extras, may be dict, or None (do nothing)

sqltrack.run_add_link(client: Client, from_id: int, kind: str, to_id: int)

Add a link between two runs.

sqltrack.run_add_metrics(client: Client, run_id: int, metrics: dict, step: int = 0, progress: float = 0.0)

Add metrics to a run.

Important

Either step or progress need to be a non-zero value to avoid overwriting existing metric values.

Note

If there is no corresponding column for metrics, this function will attempt to create them. See Defining metrics for details.

sqltrack.run_add_tags(client: Client, run_id: int, *tags: str)

Add tags to an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• tags – tags to add

sqltrack.run_from_env(client: Client) → Run

Get the Run object defined by environment variables. The experiment is defined by at least one or both of SQLTRACK_EXPERIMENT_NAME and SQLTRACK_EXPERIMENT_ID, and optionally SQLTRACK_RUN_ID.

sqltrack.run_remove_args(client: Client, run_id: int, *args: str)

Remove some arguments from an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• args – names to remove from arguments

sqltrack.run_remove_env(client: Client, run_id: int, *env: str)

Remove some environment variables from an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• env – names to remove from env

sqltrack.run_remove_extras(client: Client, run_id: int, *extras: str)

Remove some extras from an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• extras – names to remove from extras

sqltrack.run_remove_link(client: Client, from_id: int, kind: str, to_id: int)

Remove a link between two runs.

sqltrack.run_remove_tags(client: Client, run_id: int, *tags: str)

Remove tags from an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• tags – tags to remove

sqltrack.run_set_args(client: Client, run_id: int, args: 'auto' | dict | None = 'auto')

Set run arguments. See sqltrack.args.detect_args() for details on detection in auto mode. Nothing is done if no arguments are detected in auto mode.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• args – the arguments, may be 'auto' (detect arguments), dict, or None (set NULL)

sqltrack.run_set_comment(client: Client, run_id: int, comment: str | None)

Set the comment for an existing run.

sqltrack.run_set_created(client: Client, run_id: int, dt: 'auto' | datetime | None = 'auto')

Set time_created for an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• dt – the timestamp, may be 'auto' (set to now), timezone aware datetime, or None (do nothing)

sqltrack.run_set_env(client: Client, run_id: int, env: 'auto' | dict | None = 'auto')

Set run environment.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• env – the environment, may be 'auto' (set to os.environ), dict, or None (set to NULL)

sqltrack.run_set_extras(client: Client, run_id: int, extras: dict | None = None)

Set run extras.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• extras – the environment, dict, or None (set to NULL)

sqltrack.run_set_started(client: Client, run_id: int, dt: 'auto' | datetime | None = 'auto')

Set time_started for an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• dt – the timestamp, may be 'auto' (set to now), timezone aware datetime, or None (do nothing)

sqltrack.run_set_status(client: Client, run_id: int, status: str)

Set a run’s status.

sqltrack.run_set_tags(client: Client, run_id: int, *tags: str)

Set the tags of an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• tags – the tags

sqltrack.run_set_updated(client: Client, run_id: int, dt: 'auto' | datetime | None = 'auto')

Set time_updated for an existing run.

Parameters:

• client – the Client to use

• run_id – which existing run to update

• dt – the timestamp, may be 'auto' (set to now), timezone aware datetime, or None (do nothing)