Get Tables List

Retrieve the list of existing tables in the database with their record counts.

Get Tables List


Processing

Retrieves the list of existing tables in the database with their record counts. The function connects to various database types (PostgreSQL, MySQL, MariaDB, Oracle Database, Microsoft SQL Server, SQLite) and returns information about all tables including their names and row counts.

Inputs

database name
Name of the database to connect to. For SQLite, this is the path to the database file.
host (optional)
Database server hostname or IP address. Not used for SQLite databases.
port (optional)
Database server port number. Must be between 1 and 65535. Not used for SQLite databases.
username (optional)
Database username for authentication. Not used for SQLite databases.
password (optional)
Database password for authentication. Supports SecretStr for secure handling. Not used for SQLite databases.

⚠️ The function supports environment variable configuration as an alternative to direct input parameters. Input parameters are checked first; if not available, the corresponding environment variable is used.

Inputs Types

Input Types
database name Str
host Str
port Int
username Str
password SecretStr

You can check the list of supported types here: Available Type Hints.

Outputs

tables data
List of database tables with their metadata including table names and row counts. Format depends on the selected output type option.

The tables data output contains the following metadata for each table:

  • table_name: Name of the database table
  • row_count: Number of records/rows in the table

Outputs Types

Output Types
tables data DataFrame, DataRecords

You can check the list of supported types here: Available Type Hints.

Options

The Get Tables List brick contains some changeable options:

Database type
Specifies which database system to connect to. Supports PostgreSQL, MySQL, MariaDB, Oracle Database, Microsoft SQL Server, and SQLite.
Output type
Determines the format of the returned data. Can be Records (Python list), Pandas DataFrame, or Polars DataFrame.
Env variables prefix
Optional prefix for environment variable names. When specified, environment variables will be read as {prefix}_{VARIABLE_NAME} instead of just {VARIABLE_NAME}.
Verbose
Controls logging output. When enabled, provides detailed information about connection process and any errors encountered.

Environment Variables Configuration

If you don't provide database connection parameters as brick inputs, you can configure them using environment variables.

Step 1 — Create an environment file

Create an environment file in the flow project folder. You can name it anything you like (e.g. .env, db.env, or mysettings.env).

Example: db.env

# Example database environment configuration
DB_NAME=mydatabase
DB_HOST=localhost
DB_PORT=5432
DB_USERNAME=myuser
DB_PASSWORD=mypassword

💡 For SQLite, only DB_NAME is required (e.g., DB_NAME=mydb.sqlite).

Step 2 — Reference the env file in CODED FLOWS

Inside the CODED FLOWS app, use the LOAD ENV control brick to load the file you created.

  • Specify your env filename (db.env for example or whichever name you chose).
  • All defined variables become available to other any bricks in the connected graph flow.

Step 3 — Use with prefix (optional)

If you enable the "Env variables prefix" option in the LOAD ENV brick, the system will automatically prepend the prefix and an underscore to each variable.

Example with prefix MYAPP:

MYAPP_DB_NAME=mydatabase
MYAPP_DB_HOST=localhost
MYAPP_DB_PORT=5432
MYAPP_DB_USERNAME=myuser
MYAPP_DB_PASSWORD=mypassword

Bricks will then look for MYAPP_DB_NAME, MYAPP_DB_HOST, etc. instead of the default variable names.

import logging
import sqlalchemy as sa
from sqlalchemy import inspect, func, select, Table, MetaData
import pandas as pd
import polars as pl
from os import getenv
from coded_flows.types import Str, SecretStr, Int, DataFrame, DataRecords, Union

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


def _coalesce(*values):
    return next((v for v in values if v is not None))


def get_tables_list(
    database_name: Str = None,
    host: Str = None,
    port: Int = None,
    username: Str = None,
    password: SecretStr = None,
    options=None,
) -> Union[DataFrame, DataRecords]:
    brick_display_name = "Get Tables List"
    tables_data = None
    options = options or {}
    verbose = options.get("verbose", True)
    db_type = options.get("db_type", "SQLite")
    out_type = options.get("out_type", "Pandas")
    env_prefix = options.get("env_prefix", "")
    env_prefix = env_prefix + "_" if env_prefix.strip() else ""
    try:
        database_name = _coalesce(database_name, getenv(f"{env_prefix}DB_NAME"))
        if db_type != "SQLite":
            host = _coalesce(host, getenv(f"{env_prefix}DB_HOST"))
            port = _coalesce(port, getenv(f"{env_prefix}DB_PORT"))
            username = _coalesce(username, getenv(f"{env_prefix}DB_USERNAME"))
            password = _coalesce(password, getenv(f"{env_prefix}DB_PASSWORD"))
        if not database_name or (
            db_type != "SQLite"
            and (not host or not port or (not username) or (not password))
        ):
            verbose and logger.error(
                f"[{brick_display_name}] Either the env file is not defined, not referenced in the project folder, or one or more DB configuration inputs are missing."
            )
            raise ValueError(
                "Either the env file is not defined, not referenced in the project folder, or one or more DB configuration inputs are missing."
            )
        try:
            if hasattr(password, "get_secret_value"):
                password = password.get_secret_value()
            else:
                password = str(password)
        except Exception:
            password = str(password)
    except Exception as e:
        verbose and logger.error(
            f"[{brick_display_name}] Either the env file is not defined, not referenced in the project folder, or one or more DB configuration inputs are missing."
        )
        raise ValueError(
            "Either the env file is not defined, not referenced in the project folder, or one or more DB configuration inputs are missing."
        )
    try:
        if port:
            port = int(port)
    except (ValueError, TypeError):
        verbose and logger.error(
            f"[{brick_display_name}] Port must be an integer, got {port!r}"
        )
        raise ValueError(f"Port must be an integer, got {port!r}")
    if port and (not 1 <= port <= 65535):
        verbose and logger.error(
            f"[{brick_display_name}] Port must be between 1 and 65535, got {port}"
        )
        raise ValueError(f"Port must be between 1 and 65535, got {port}")
    verbose and logger.info(
        f"[{brick_display_name}] Retrieving list of tables from {db_type} database."
    )
    driver_map = {
        "Postgres": "postgresql+psycopg",
        "MySQL": "mysql+pymysql",
        "MariaDB": "mariadb+pymysql",
        "Oracle Database": "oracle+oracledb",
        "Microsoft SQL Server": "mssql+pymssql",
        "SQLite": "sqlite",
    }
    if db_type == "SQLite":
        if host or username or password:
            verbose and logger.warning(
                f"[{brick_display_name}] Ignoring host, username, password for SQLite."
            )
        url = f"{driver_map[db_type]}:///{database_name}"
    else:
        if not all([host, str(port), username, password, database_name]):
            verbose and logger.error(
                f"[{brick_display_name}] Missing required connection parameters for {db_type}."
            )
            raise ValueError(
                f"[{brick_display_name}] Missing required connection parameters for {db_type}."
            )
        url = f"{driver_map[db_type]}://{username}:{password}@{host}:{port}/{database_name}"
    verbose and logger.info(f"[{brick_display_name}] Connection URL constructed.")
    try:
        engine = sa.create_engine(url)
        verbose and logger.info(f"[{brick_display_name}] Engine created.")
        inspector = inspect(engine)
        table_names = inspector.get_table_names()
        data = []
        metadata = MetaData()
        with engine.connect() as connection:
            for table in table_names:
                table_obj = Table(table, metadata)
                count_query = select(func.count()).select_from(table_obj)
                row_count = connection.scalar(count_query)
                data.append({"table_name": table, "row_count": row_count})
        if out_type.lower() == "pandas":
            tables_data = pd.DataFrame(data)
        elif out_type.lower() == "polars":
            tables_data = pl.DataFrame(data)
        else:
            tables_data = data
        engine.dispose()
        verbose and logger.info(
            f"[{brick_display_name}] Tables list retrieved successfully. Tables found: {(len(tables_data) if tables_data is not None else 0)}"
        )
    except Exception as e:
        verbose and logger.error(
            f"[{brick_display_name}] An error occurred while retrieving tables list: {str(e)}"
        )
        raise
    return tables_data

Brick Info

version v0.1.0
python 3.10, 3.11, 3.12, 3.13
requirements
  • cryptography
  • sqlalchemy
  • psycopg[binary]
  • pymysql
  • pymssql
  • pandas
  • polars
  • oracledb