pip install api2db

api2db is designed to make building data-pipelines easy.

because **** messy data and scope creep

Note

Project requirements changed? No problem.

API got updated? Easy.

Changing databases? Change 1 line of code.

Multiple storage targets? Done.

Stop focusing on collecting data, and start focusing on using it.

Use api2db to:

• Collect data from any API

• Perform pre-processing on data

• Clean and extract features from data

• Perform post-processing on data

• Store data

api2db supports any/all of the following concurrently:

• Streaming data live to

  • Local storage of data in files using Parquet, pickle, JSON, and CSV format

  • MySQL

  • Bigquery

  • MariaDB

  • PostgreSQL

  • Omnisci

• Storing data periodically to

  • MySQL

  • Bigquery

  • MariaDB

  • PostgreSQL

  • Omnisci

api2db is currently adding support for:

• Oracle

• Amazon Aurora

• Microsoft SQL Server

• Firebase RTDB

• Don’t see your database? Submit a feature request.. or even better add it. api2db is open-source.

Contents

• Installation and Quickstart
  • Installation
  • Quickstart
  • Using the lab to build ApiForms
  • Importing data
  • Performing pre-processing on data
  • Extracting features from data
  • Performing post-processing on data
  • Exporting data from the lab to a collector
  • Setting up an authentication file for database targets
  • Setting up a stream target for live data
  • Setting up a store target for data
  • Registering a collector to run
  • Running the application
• Examples
  • Shell Commands
  • Pre-processing
  • Extracting data-features
  • Post-processing
• The future of api2db
  • Future Plans
• api2db
  • api2db package

The creator of api2db is currently searching for a job. He graduates with a bachelors in CS May 15th

Contact him by emailing tristenharr@gmail.com

Indices and tables

• Index

• Module Index

• Search Page