API microservice, using Python and FastAPI, for testing files against industry schemas. Data validation uses Pandera.

Development Instructions

Clone this repo and install Python. From the root directory of this repo, run:

python -m venv ./venv #Create your python virtual environment

#WINDOWS
venv\Scripts\activate.bat
#LINUX
. venv/bin/activate

pip install --no-cache-dir --upgrade -r requirements.txt #Install project dependancies 

cd into the root directory with main.py

uvicorn main:app --reload #Start the Fast API server

Typically hosted at http://localhost:8000.

Folder Structure

• Fast API endpoints are hosted in main.py in the root directory
• /schemamodels/models contain the Pandera schema model files
• /schemamodels/models/controlled_lists contains the controlled list as a .csv to be updated
• updated or new controlled lists are imported into the relevant Pandera schema

Production Instructions

A docker file and github workflow is included within this repository. When a release is created, the docker container is built and submitted to dockerhub.

It is assumed you already have accounts for Dockerhub and Github, along with a Docker instance set up for the built container to run on. If any information is needed on how ecosystem2 handles hosting, please reach to us via our website or email.

Seting up dockerhub access token

• Once logged into Dockerhub, select the downward triangle next to your username on the far right of your screen.

  

• A dropdown should present itself, use this to enter your account settings

  

• On the sidebar to the left, Select Security

  

• To the right of the Access Tokens heading, Click on the blue New Access Token button

  

• A popup should now appear in the centre of the screen, Set the appropriate description and permissions

  

• Your new token should now be created, Copy the token

  

Setting up dockerhub credentials

• Below the repository name, in the top left corner of the screen, there is a navigation bar; Select Settings

  

• You should be presented with a page consisting with a sidbar and the repositories settings, Select Secrets and variables

  

• With the sub-section expanded, Select Actions

  

• This section should already have two repository secrets, DOCKER_USERNAME and DOCKER_PASSWORD

  

• If this is not the case, please add them
• Assuming both secrets are present, Select the pencil icon to edit each secret

Building docker container

• On the left sidebar, Select Releases
• Select Draft New Release
• Type in a new version number into tag field and select "Create new tag on publish".
• Set the target branch to main
• Either type the title and descript manually or select "Generate release notes"
• Press the green button that says "Publish release"
• Wait for the build to complete

Feel free to use the included docker-compose file, please note that it uses Traefik in order to create routes and assign SSL certificates.

Screenshot	Description
	Current state of the webapp.

TODO

• Upload Working example
• Upload Demonstartion video & GIF
• Share with collaborator

Contributions

Pull requests are appreciated!