starfish-py

Table of Contents

• About
• Features
• Prerequisites
• Quickstart
• Environment variables
• Testing
• New Version
• Mailing Lists
• Maintainers
• License

About

Starfish is an open-sourced developer toolkit for the data economy. Available in flavours of Java, Python, and Clojure, it allows developers, data scientists and enterprises to create, interact, integrate and manage a data supply line through standardised and simple-to-use APIs.

Based on an underlying data ecosystem standard, Starfish provides high-level APIs for common tasks within the data economy, for example, registering/publishing an asset, for subsequent use in a data supply line. In this case, an asset can be any data set, model or data service. The high-level API also allows developers to invoke operation on an asset, e.g. computing a predictive model or anonymising sensitive personal information, among other capabilities.

Starfish works with blockchain networks, such as Convex, and common web services through agents, allowing unprecedented flexibility in asset discovery and data supply line management.

Starfish-py provides user access and tools to work with the Convex Network, delegating certain functions via the python library convex-api-py.

While we strive to deliver code at a high quality, please note, that there exist parts of the library that still need thorough testing. Contributions – whether it is in the form of new features, better documentation or tests – are welcome.

Prerequisites

Python >= 3.10

Development

1. Clone this repo

   clone https://github.com/datacraft-dsc/starfish-py.git
   cd starfish-py

2. Set up a virtual environment

   virtualenv venv
   source venv/bin/activate

3. Install package requirements for starfish

   make install

4. Run the unit tests, without any supporting software/libraries outside of starfish

   make test_unit

5. Create the local testing environment using datacraft-chain.

   In a separate terminal session you need to clone and checkout datacraft-chain repository, by doing the following:

   git clone https://github.com/datacraft-dsc/datacraft-chain.git
   cd datacraft-chain
   ./start_datacraft_chain.sh test

6. Run the integration tests

   pytest tests/integration

7. Run the all tests

   make tests

Documentation

1. Build Sphinx documentation

   To build the Sphinx auto documentation, you need to do the following:

   make docs

The documentation for this repo.

Testing

Automatic tests are setup via github actions. Our test use pytest framework. The testing uses a datacraft-chain docker image and surfer server. See github actions for testing

New Version

The bumpversion.sh script helps to bump the project version. You can execute the script using as first argument {major|minor|patch} to bump accordingly the version.

Mailing Lists

• starfish-qa -- General questions regarding the usage of Starfish.

Release Process

See Release Process

Maintainers

Developer Datacraft team

License

Copyright 2018-2021 Datacraft Pte. Ltd.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.