search

os-climate / aicoe-osc-demo

This repository is the central location for the demos the ET data science team is developing within the OS-Climate project. This demo shows how to use the tools provided by Open Data Hub (ODH) running on the Operate First cluster to perform ETL, create training and inference pipelines.
Apache License 2.0
10 stars 24 forks source link

Add transformer implementation of relevance task #216

Closed Shreyanand closed 1 year ago

Shreyanand commented 1 year ago

This PR adds transformer implementation and distilbert base model for the relevance task. These notebooks will enable us to experiment with sparse models.

Reference Issues: Closes #221 Addresses #184

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

Shreyanand commented 1 year ago

Hi @Shreyanand , I am trying to run this notebook. I am encountering some minor bugs like "no module error" for sparsezoo, evaluate and datdaset. Is it possible to include them in the dependencies file? wdyt?

Good observation, I tried adding those to dependency files but some of them will cause an error with the farm notebooks as those notebooks rely on older versions of these packages. We need overlays for this problem but it'll take a while so for our experiments we'd have to pip install them for the time being.

suppathak commented 1 year ago

/lgtm

Shreyanand commented 1 year ago

/approve

sesheta commented 1 year ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Shreyanand

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/os-climate/aicoe-osc-demo/blob/master/OWNERS)~~ [Shreyanand] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment