[C4GT] File Validator for cQube

[invinayak007]

Project Details:

This project aims to develop a File Validation utility with a user-friendly interface for cQube. The utility will perform checks files to be uploaded to ensure that the data meets the required specifications and is accurate and complete. Basically the Event file has to be validated before uploading that to cQube.

The key features to be implemented as part of this project are:

• User-friendly interface: The utility should have an intuitive and user-friendly interface that allows the end user to easily upload and validate their files.
• Validation rules: The File Validator utility should include validation rules that cover common file-related issues.
  • Format : Format of the data matches as per the spec defined earlier which is available in cQube
  • Completeness : All the required fields are filled
  • Accuracy
    • Dimension values : Dimension values are validated against the master data available in cQube
    • Metric values : Metric values should be within a defined range
• Custom validation rules: The utility should allow the end user to define custom validation rules specific to their data. This would enable them to check for errors or inconsistencies that are unique to their data.
  • Ranges for certain fields
  • Pattern of a particular field
• Error reporting: The File Validator utility should provide clear and detailed error messages when a file fails validation. This would help the end user identify and fix any issues with their data.

Learning Path:

Complexity: Moderate Risk/Exploratory: Low Skills Required: Python, familiarity with data validation techniques. Name of Mentor: Vinayak Inamadar Project size (in hours): 150-200 hours

Product Set Up:

To set up the project on a system, follow these guidelines:

• Install the required programming language and libraries, such as Python and their respective data validation libraries.
• Fork the project
• Clone the forked project repository from GitHub.
• Set up the environment variables and configurations required to run the utility.

For more details, go through https://github.com/Code4GovTech/C4GT

Acceptance Criteria:

The acceptance criteria for this project are:

• The utility should be able to validate files uploaded by the end user and identify any errors or inconsistencies.
• The utility should provide clear and detailed error messages when a file fails validation.
• The utility should have an intuitive and user-friendly interface that allows the end user to easily upload and validate their files.

Out of Scope Elements:

The following elements are out of scope for this project:

• Data cleaning and normalization.
• Handling of large files that require distributed processing.
• Integration with other systems or tools.

Key milestones for this project are:

• Development of the File Validator utility with basic validation rules
• Implementation of custom validation rules and error reporting
• Coming up with UI for easy usage
• Testing and debugging of the utility

[MridulMohanta19]

I am interested in contributing in this project as I am fairly proficient in the language and I want to work on this project.

[prakharrathi25]

I am also interested in this project. I am quite familiar with Python and it's related libraries and have contributed to open source projects using this tech stack.

[Srushhhti]

I am also interested in this project as I am well familiar with python and its libraries. I have worked upon some simple projects online and I would like to contribute here.

[lohith1392]

I am interested in this project. I am quite familiar with Python and its usage in the backend like in Django, flask, and Machine learning. I want to work on this project and can include other features if time permits like duplicate detection and referential integrity.

[mahi01agarwal]

I am very interested in contributing to it. After reviewing the project description and codebase, I believe my skills and experience align well with the goals of the project. I would love the opportunity to work with you and your team to help bring this project to the next level.

[pranavsusarla]

I believe I will be able to contribute to this project! Looking forward to it😄

[Raghavi7124]

I'm interested in contributing to this project! As I believe that my skills would get aligned with this project!!!

[invinayak007]

Hi All, Good to know the response so far. Please start working on the proposals. If you have any queries please post it here. Thanks

[Srushhhti]

Hey, can you please provide me with the codebase for this project.

[harshinirhsv]

hi ,I'm interested in contributing to this project

[harshinirhsv]

we can start writing the proposal directly or need to resolve any issues?

[Komal-99]

I am also interested in this project as I have worked upon various projects related to python and data processing in ML field I would like to contribute here .

[invinayak007]

Thanks for your interest. Below is some detail on proposal writing.

What proposal denotes? The basic motive behind the proposal is to analyse how proficient the student's understanding is regarding the project, along with their past work experiences and understanding about an open source.

How to start with proposal writing? You can check this template for the proposal writing, can include these pointers in your proposal: https://github.com/Code4GovTech/C4GT/wiki/Sample-Proposal

Before writing proposal, one should be clear with the implementation and the requriements of the projects, so try to interact with mentors check the previous discussions on the project channel of your interest, you can also interact with other contributors and can raise the joint PRs.

Note: After writing your proposal before submission, you can share it with your respective mentors of the project, they can share their feedback over it and you can incoporate that in your proposal.

All the best, Keep learning!

[Komal-99]

Thanks for your interest. Below is some detail on proposal writing.

What proposal denotes? The basic motive behind the proposal is to analyse how proficient the student's understanding is regarding the project, along with their past work experiences and understanding about an open source.

How to start with proposal writing? You can check this template for the proposal writing, can include these pointers in your proposal: https://github.com/Code4GovTech/C4GT/wiki/Sample-Proposal

Before writing proposal, one should be clear with the implementation and the requriements of the projects, so try to interact with mentors check the previous discussions on the project channel of your interest, you can also interact with other contributors and can raise the joint PRs.

Note: After writing your proposal before submission, you can share it with your respective mentors of the project, they can share their feedback over it and you can incoporate that in your proposal.

All the best, Keep learning!

Hi , Any communication channel for the project ?

[invinayak007]

Please go through https://unstop.com/competitions/code-for-govtech-2023-samagra-governance-675801

[masterismail]

Hi @invinayak007 I've written a proposal for this project would you let me know the best channel to connect with you to get a review for the same.

[Ashutosh27102002]

Highly Interested, have done a similar project for submission on Yukti Portal (Government initiative to promote start-ups) and its currently in for Round 3

[Arjitm26]

Interested to contribute for the project!

[yash-0211]

Looking forward to contributing to this project :)

[moreatharvaa]

already familiar with the language and would love to contribute!

[jainendra001]

Hi @invinayak007 ,

I'm Jainendra Tripathy, and I am interested in contributing to this project. I'm an AI and ML developer with good hands-on experience in Python. I have a strong understanding of Python, Data validation techniques., and I'm always looking for new ways to learn and grow. I'm also a quick learner and always willing to help. I'm available to contribute . I'm eager to get started and make a difference. I'm confident I can help you take this project to the next level. If you're interested in having me contribute, please let me know. I'm happy to answer any questions you have. And I am also applying to the C4GT too please consider my application.

Thanks, Jainendra Tripathy

[invinayak007]

As many of you were asking about more clarity, I am adding few things here

1. End user will run the utility (possibly a flask application or you can come up with other ideas)
2. User will select the particular program(which cQube supports) for which the file needs to be validated.
3. Based on the program selected, the utility should fetch the specification/schema which is stored in DB (in JSON).
4. Specific validations as per the specification/schema and other generic validations as discussed in the issue description should happen.
5. You can think of some more UI components to help user define custom validations and process (ex: user can enter column name and range for the values)
6. You can fine more about schemas here
7. Finally it should prepare a report indicating the validation issues.

Let me know if you have any further questions on this.

[VivekPrakashNitH]

I am interested in this project as i very familiar with this language and has built many ML related projects on this . Link to one of my end project :- https://github.com/VivekPrakashNitH/open-air-gym and many more ML and AI related projects like this

[AdityaSen-1606]

Hi @invinayak007, I am interested in solving this issue. I have the skills to solve this issue and am confident that I will add significant value to this project.

[VivekPrakashNitH]

why did you message me? I am not the Mentor please message in discord

On Fri, Jun 9, 2023 at 12:25 PM Mugen @.***> wrote:

Hi @invinayak007 https://github.com/invinayak007, I am interested in solving this issue. I have the skills to solve this issue and am confident that I will add significant value to this project.

— Reply to this email directly, view it on GitHub https://github.com/Sunbird-cQube/InputFileValidator/issues/1#issuecomment-1584068154, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYB23HL4VD3GFT7JLFG627TXKLCFNANCNFSM6AAAAAAX6BHSR4 . You are receiving this because you commented.Message ID: @.***>

[Fayouzz]

Dear Mentor, I have submitted my proposal for "[C4GT] File Validator for cQube #1". Looking forward to learning and contributing :) Thank you!