API Documentation

dbgdb

Data pipeline utilities to help you get data into and out of your database.

dbgdb.db.postgres

This module contains utility functions to help when working with PostgreSQL databases.

dbgdb.db.postgres.connect(url: str, dbname: str = None, autocommit: bool = False)

Create a connection to a Postgres database.

Parameters:

• url – the Postgres instance URL
• dbname – the target database name (if it differs from the one specified in the URL)
• autocommit – Set the autocommit flag on the connection?

Returns:

a psycopg2 connection

dbgdb.db.postgres.create_db(url: str, dbname: str, admindb: str = 'postgres')

Create a database on a Postgres instance.

Parameters:

• url – the Postgres instance URL
• dbname – the name of the database
• admindb – the name of an existing (presumably the main) database

Returns:

dbgdb.db.postgres.create_extensions(url: str)

Create the necessary database extensions.

Parameters:url – the URL of the database instance

dbgdb.db.postgres.create_schema(url: str, schema: str)

Create a schema in the database.

Parameters:

• url – the URL of the database instance
• schema – the name of the schema

dbgdb.db.postgres.db_exists(url: str, dbname: str = None, admindb: str = 'postgres') → bool

Does a given database on a Postgres instance exist?

Parameters:

• url – the Postgres instance URL
• dbname – the name of the database to test
• admindb – the name of an existing (presumably the main) database

Returns:

True if the database exists, otherwise False

dbgdb.db.postgres.drop_schema(url: str, schema: str)

Drop a schema from the database.

Parameters:

• url – the URL of the database instance
• schema – the name of the schema

dbgdb.db.postgres.schema_exists(url: str, schema: str)

Does a given schema exist within a Postgres database?

Parameters:

• url – the Postgres instance URL and database
• schema – the name of the schema

Returns:

True if the schema exists, otherwise False

dbgdb.db.postgres.select_schema_tables(url: str, schema: str) → Iterable[str]

Select the names of the tables within a given schema.

Parameters:

• url – the URL of the dat
• schema – the name of the schema

dbgdb.cli

This is the entry point for the command-line interface (CLI) application. It can be used as a handy facility for running the task from a command line.

Note

To learn more about Click visit the project website. There is also a very helpful tutorial video.

To learn more about running Luigi, visit the Luigi project’s Read-The-Docs page.

class dbgdb.cli.Info

Bases: object

This is an information object that can be used to pass data between CLI functions.

__init__()

Initialize self. See help(type(self)) for accurate signature.

dbgdb.cli.pass_info(f)

pylint: disable=invalid-name

dbgdb.cli.run(tasks: Iterable[luigi.task.Task], info: dbgdb.cli.Info)

Run tasks on the local scheduler.

Parameters:

• tasks – the tasks to run
• info – the Info object containing other parameters

dbgdb.ogr.postgres

This module contains wrapper functions for work that would generally be handled by OGR/GDAL.

class dbgdb.ogr.postgres.OgrDrivers

Bases: enum.Enum

These are the supported OGR drivers.

GeoPackage = 'GPKG'

PostGIS = 'PostgreSQL'

Spatialite = 'SQLITE'

dbgdb.ogr.postgres.extract(outdata: pathlib.Path, schema: str = 'imports', url: str = 'postgresql://postgres@localhost:5432/postgres', driver: dbgdb.ogr.postgres.OgrDrivers = <OgrDrivers.Spatialite: 'SQLITE'>)

Extract a schema from a PostgreSQL database to a file geodatabase.

Parameters:

• outdata – the path to the output
• schema – the schema to export
• url – the URL of the Postgres database instance
• driver – the OGR driver to use

dbgdb.ogr.postgres.load(indata: pathlib.Path, url: str = 'postgresql://postgres@localhost:5432/postgres', schema: str = 'imports', overwrite: bool = True, progress: bool = True, use_copy: bool = True, driver: dbgdb.ogr.postgres.OgrDrivers = <OgrDrivers.PostGIS: 'PostgreSQL'>)

Load a file geodatabase (GDB) into a Postgres database.

Parameters:

• indata – the path to the file geodatabase
• url – the URL of the Postgres instance
• schema – the target schema
• overwrite – Overwrite existing data?
• progress – Show progress?
• use_copy – Use COPY instead of INSERT when loading?
• driver – the OGR driver to use

dbgdb.targets.postgres

This module contains PostgreSQL targets.

class dbgdb.targets.postgres.PgSchemaTarget(url: str, schema: str, dne: bool = False)

Bases: luigi.target.Target

This is a target that represents a file geodatabase (GDB).

__init__(url: str, schema: str, dne: bool = False)

Parameters:

• url – the path to the file GDB
• schema – the target schema
• dne – (“does not exist”) this should be True if the target’s task is considered to be complete if the schema does not exist

connect()

Get a connection to the database.

Returns:a connection to the database

exists() → bool

Does the file target schema exist?

Returns:True if the file geodatabase exists, otherwise False

dbgdb.tasks.postgres.drop

This module contains the PgDropSchemaTask task which you can use to drop a schema.

class dbgdb.tasks.postgres.drop.PgDropSchemaTask(*args, **kwargs)

Bases: luigi.task.Task

This task loads a file geodatabase into a database instance.

Variables:

• url – the URL of the target database
• schema – the target schema

output() → dbgdb.targets.postgres.PgSchemaTarget

This task returns a PgSchemaTarget that points to the target schema where the GDB was loaded.

Returns:the PostgreSQL schema target

requires()

This task has no requirements.

Returns:an empty iteration

run()

Run the task.

schema = <luigi.parameter.Parameter object>

url = <luigi.parameter.Parameter object>

dbgdb.tasks.postgres.extract

This module contains the ExtractTask task which you can use to extract data from your database instance.

class dbgdb.tasks.postgres.extract.PgExtractTask(*args, **kwargs)

Bases: luigi.task.Task

This task loads a file geodatabase into a database instance.

Variables:

• url – the URL of the target database
• schema – the target schema
• outdata – the path to which data should be exported
• driver – the driver to use for exporting data

driver = <luigi.parameter.EnumParameter object>

outdata = <luigi.parameter.Parameter object>

output() → luigi.local_target.LocalTarget

This task returns a PgSchemaTarget that points to the target schema where the GDB was loaded.

Returns:the PostgreSQL schema target

requires()

This task has no requirements.

Returns:an empty iteration

run()

Run the task.

schema = <luigi.parameter.Parameter object>

url = <luigi.parameter.Parameter object>

dbgdb.tasks.postgres.load

This module contains the LoadTask task which you can use to load a file geodatabase into your database instance.

class dbgdb.tasks.postgres.load.PgLoadTask(*args, **kwargs)

Bases: luigi.task.Task

This task loads a file geodatabase into a database instance.

Variables:

• url – the URL of the target database
• schema – the target schema
• indata – the path to the input data

indata = <luigi.parameter.Parameter object>

output() → dbgdb.targets.postgres.PgSchemaTarget

This task returns a PgSchemaTarget that points to the target schema where the data was loaded.

Returns:the PostgreSQL schema target

requires()

This task has no requirements.

Returns:an empty iteration

run()

Run the task.

schema = <luigi.parameter.Parameter object>

url = <luigi.parameter.Parameter object>