petrobras / 3W

Promotes development of ML algorithms for early detection and classification of undesirable events in offshore oil wells.
Apache License 2.0
326 stars 79 forks source link

feat: Backlog - docker, git lfs, review of strategy and a new site for data visualization #120

Open Arthur-Llevy opened 3 weeks ago

Arthur-Llevy commented 3 weeks ago

What was done: Some of the suggestions from the 3W repository backlog, plus some extra contributions

Backlog - Evaluate and, if applicable, start using Git LFS

Since one of the Backlog items was a possible use of Git LFS, I decided to bring a quick guide for developers/project collaborators on how to use Git LFS to version large files, considering that we are working with data.

Backlog - Evaluate the use of Docker to facilitate the use of the 3W Toolkit and the approval of contributions;

Docker was implemented to facilitate the installation of the project dependencies (Python, Conda and Jupyter Notebook), in order to reduce the time spent on setting up the environment. Just run the following command "docker compose up --build". For the time being, Jupyter Notebook only runs on Linux-based operating systems, but this does not affect the functioning of 3View, which will be discussed later; it works normally on any system with a browser.

Backlog - Review strategy for specifying the virtual environment (environment.yml)

Suggestions for resolving this issue: Fix the Python version to 3.10: This allows patch updates (e.g. 3.10.7, 3.10.8) without breaking compatibility, ensuring flexibility and maintaining stability. Perform minor updates (bug fixes, security patches). Add pip for extra dependencies: This allows packages not available in conda to be installed via PyPI, while reducing potential conflicts by ensuring that conda dependencies are installed first.

Extra contribution

The "3view" website has been created, which serves to visualize the data and the list of people who contributed to 3w. In it, you can visually choose the database you want to view. The option to download the database file that is being displayed has also been added. To run 3view, you need to run docker. To do this, run:

docker compose -up --build

After that, simply access "localhost:5000" in your browser when the service is live

Captura de tela de 2024-10-19 16-29-42 Captura de tela de 2024-10-19 16-29-53

Photos of the project: Captura de tela em 2024-10-13 11-25-11 Captura de tela em 2024-10-13 11-25-14 Captura de tela em 2024-10-13 11-25-22