Isaiahensley / Aquatic-RIG

Our Senior Capstone project focuses on developing a Streamlit website dedicated to the visualization of aquatic NetCDF datasets. Aquatic data is inherently complex, being both spatiotemporal—capturing information over different times and space. Our website will let users alternate through time and depth to give comprehensive visuals for their data.
https://aquaticrig-develop.streamlit.app/
0 stars 1 forks source link

Creating a Workflow for Linting and Formatting #51

Open blackm61 opened 8 months ago

blackm61 commented 8 months ago

Description: This workflow automates the process of setting up a Python environment, installing dependencies, and checking the code quality and formatting of a Python project whenever changes are pushed to specific branches of the repository.

Objective: The objective was to check all the codes that's been pushed in the repository to pass through lynting and code formatting checks.

Steps to reproduce: The main goal is still to integrate few more testing and more workflow. I haven't encountered any issue but lack of ideas for the workflow.

Expected Behaviour: The YAML config file automates the process defined.

Actual Behaviour: Its helping me to maintain the consistent and quality standards for code.ting and code formatting checks.

spaude11 commented 8 months ago

https://github.com/Isaiahensley/Aquatic-RIG/tree/main/.github/workflows

I will add all the workflows that is required for integration and deployment.

spaude11 commented 8 months ago

This workflow checks if the code integrates with the rest of the project. Its set up in develop branch and we will check if any pushes happens in develop branch it will check if the code is formatted correctly.

Triggers: It triggers on every push to the main branch. Jobs: It defines a single job named build. Steps: Checkout Repository: This step checks out your repository's code. Set up Python: Sets up the Python environment using the specified version. Install dependencies: Installs project dependencies using pip. Lint code: pylint Check code formatting: code formatting checkers

spaude11 commented 8 months ago

I pulled the code from develop branch and made some changes into the feature/pipeline integration branch and integrated the workflow with the feature/pipeline integration branch

Isaiahensley commented 8 months ago

Have you checked out streamlit test.

https://docs.streamlit.io/library/api-reference/app-testing

it’s a built in testing framework for Streamlit code

Isaiahensley commented 8 months ago

@spaude11 Added the code you worked on yesterday. We will make sure to include our code for every standup from now on. We'll also want to have the code we share explained so it's easy to understand.

python-package.yml

name: Continuous Integration

on:
  push:
    branches:
      - feature/pipeline-integration  

jobs:
  build:
    name: Build and Check
    runs-on: ubuntu-latest  

    steps:
    - name: Checkout Repository
      uses: actions/checkout@v2

    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: 3.9.6 

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt  # Assuming you have a requirements.txt file

    - name: Lint code
      run: |
        # Example command to run linting tool (adjust according to your project)
        pylint your_code_directory/

    - name: Check code formatting
      run: |
        # Example command to check code formatting (adjust according to your project)
        black --check your_code_directory/

test_datasetmanagement_page.py

import streamlit as st
from datetime import datetime
import pytest
from unittest.mock import patch

from datasetmanagement_page.py import datasetmanagement_page

@pytest.fixture
def streamlit_dg_fixture():
    with patch("streamlit.DeltaGenerator") as mock_dg:
        yield mock_dg

def test_datasetmanagement_page(streamlit_dg_fixture):
    # Arrange
    mock_dg = streamlit_dg_fixture()
    # Simulate file upload
    mock_file_upload = "mocked_file_contents"
    with patch("streamlit.file_uploader", return_value=mock_file_upload):
        # Act
        datasetmanagement_page()

    # Assert
    mock_dg.title.assert_called_with("Dataset Management")
    mock_dg.file_uploader.assert_called_with("Upload dataset", type=["nc"])

    mock_dg.selectbox.assert_called_with(
        "How would you like this dataset visualized?",
        ("Scatter Plot", "Quiver", "Heat Map", "Contourf")
    )
# Simulate sliders
mock_dg.slider.assert_any_call("Time", value=datetime(2020, 1, 1, 9, 30), format="MM/DD/YY - hh:mm")
mock_dg.slider.assert_any_call("Depth", 0, 100)
spaude11 commented 8 months ago

made some changes in the linting as well as put the directory, here's the updated code. name: Continuous Integration

on: push: branches:

jobs: build: name: Build and Check runs-on: ubuntu-latest

steps:
- name: Checkout Repository
  uses: actions/checkout@v2

- name: Set up Python
  uses: actions/setup-python@v2
  with:
    python-version: 3.9.6 

- name: Install dependencies
  run: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt 

- name: Lint code
  run: |
    pylint .

- name: Check code formatting
  run: |
    black --check .
spaude11 commented 8 months ago

Objective: The objective was to check all the codes that's been pushed in the repository to pass through lynting and code formatting checks.

Steps to reproduce: The main goal is still to integrate few more testing and more workflow. I haven't encountered any issue but lack of ideas for the workflow.

Description: This workflow automates the process of setting up a Python environment, installing dependencies, and checking the code quality and formatting of a Python project whenever changes are pushed to specific branches of the repository.

Expected Behaviour: The YAML config file automates the process defined. Actual Behaviour: Its helping me to maintain the consistent and quality standards for code.

Code Snippet: ` name: Continuous Integration

on:
  push:
    branches:
      - feature/pipeline-integration 
      - feature/dataset-management-page
jobs:
  build:
    name: Build and Check
    runs-on: ubuntu-latest  
   steps:
    - name: Checkout Repository
      uses: actions/checkout@v2

    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: 3.9.6 

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt 

    - name: Lint code
      run: |
        pylint .

    - name: Check code formatting
      run: |
        black --check .`

Environment Details: Github ChatGPT

will create a new workflow and one test for my page.

spaude11 commented 8 months ago

Objective: The objective was to check all the codes that's been pushed in the repository to pass through lynting and code formatting checks.

Steps to reproduce: The main goal is still to integrate few more testing and more workflow. I encounter the error while working on it, since I am not able to get the workflow to run through all the branches as mentioned in the YAML.

Screenshot 2024-02-21 at 10 20 23 am

Description: This workflow automates the process of setting up a Python environment, installing dependencies, and checking the code quality and formatting of a Python project whenever changes are pushed to specific branches of the repository. I am getting error and I am trying to understand what the error is and how to resolve it.

Expected Behaviour: The YAML config file automates the process defined.

Actual Behaviour: Its not running through all the branches and I am trying to figure out the issue.

Code Snippet: ` name: Continuous Integration

on:
  push:
    branches:
      - feature/pipeline-integration 
      - feature/dataset-management-page
jobs:
  build:
    name: Build and Check
    runs-on: ubuntu-latest  
   steps:
    - name: Checkout Repository
      uses: actions/checkout@v2

    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: 3.9.6 

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt 

    - name: Lint code
      run: |
        pylint .

    - name: Check code formatting
      run: |
        black --check .`

Environment Details: Github ChatGPT

Issue Resolution: I will update the issue resolution, after resolving the error.

spaude11 commented 8 months ago
Screenshot 2024-02-25 at 7 13 46 pm
spaude11 commented 8 months ago

Description: This workflow automates the process of setting up a Python environment, installing dependencies, and checking the code quality and formatting of a Python project whenever changes are pushed to specific branches of the repository.

Objective: The objective was to check all the codes that's been pushed in the repository to pass through lynting and code formatting checks.

Steps to reproduce: The main goal is still to integrate few more testing and more workflow. Encountered the build fails on few of the works.

Image

Expected Behaviour: The YAML config file automates the process defined. The workflow should give the feedback based on the format of the python code. It pro

Code Snippet:

`name: Pylint

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.8", "3.9", "3.10"]
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v3
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install pylint
    - name: Analysing the code with pylint
      run: pylint $(git ls-files '*.py') this is my pylint fil e

`

Actual Behaviour: Its helping me to maintain the consistent and quality standards for code.ting and code formatting checks.

Isaiahensley commented 8 months ago

You showed me these pylint tests running and it looks great. After we make pushes to the develop branch we could try to run these tests on it.

spaude11 commented 7 months ago

Description This workflow automates the process of setting up a Python environment, installing dependencies, and checking the code quality and formatting of a Python project whenever changes are pushed to specific branches of the repository.


Objective: The objective was to check all the codes that's been pushed in the repository to pass through lynting and code formatting checks.


Steps to reproduce: The main goal is still to integrate few more testing and more workflow. Encountered the build fails on few of the works.

Image

Expected Behaviour: The YAML config file automates the process defined. The workflow should give the feedback based on the format of the python code.

Code Snippets

name: Pylint

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.8", "3.9", "3.10"]
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v3
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install pylint
    - name: Analysing the code with pylint
      run: pylint $(git ls-files '*.py')

Actual Behaviour: Its helping me to maintain the consistent and quality standards for code.ting and code formatting checks.

Image


Environment Details:

Github ChatGPT