[BUG]: requirements.txt in log-parsing-models for training and tuning models folder

nyck33 commented 1 year ago

Version

23.11

Which installation method(s) does this occur on?

Docker

Describe the bug.

When I cd to

coder ➜ .../morpheus/models/training-tuning-scripts/log-parsing-models $ ls
log-parsing-training.ipynb  requirements.txt  resources
coder ➜ .../morpheus/models/training-tuning-scripts/log-parsing-models $ pip install -r requirements.txt
Defaulting to user installation because normal site-packages is not writeable
ERROR: Invalid requirement: 'onnx=1.14.0' (from line 1 of requirements.txt)
Hint: = is not a valid operator. Did you mean == ?
coder ➜ .../morpheus/models/training-tuning-scripts/lo

I open requirements.txt and see

onnx=1.14.0
seqeval==1.2.2
transformers==4.22.2

Minimum reproducible example

Please see above.

Relevant log output

Just a minor typo

Full env printout

(morpheus) coder ➜ /workspaces/morpheus/external $ ls
morpheus-visualizations  utilities
(morpheus) coder ➜ /workspaces/morpheus/external $ cd utilities
(morpheus) coder ➜ /workspaces/morpheus/external/utilities $ ls
(morpheus) coder ➜ /workspaces/morpheus/external/utilities $ ls
(morpheus) coder ➜ /workspaces/morpheus/external/utilities $

Not sure why it's not there, in the dev container.

Other/Misc.

No response

Code of Conduct

[X] I agree to follow Morpheus' Code of Conduct
[X] I have searched the open bugs and have found no duplicates for this bug report

jarmak-nv commented 1 year ago

Hi @nyck33!

Thanks for submitting this issue - our team has been notified and we'll get back to you as soon as we can! In the mean time, feel free to add any relevant information to this issue.

nyck33 commented 1 year ago

Actually I made a conda env with the requirements.txt in the log-parsing-models folder but it's first cell requires torch which is not on that file

from seqeval.metrics import classification_report,accuracy_score,f1_score
from transformers import BertForTokenClassification
from tqdm import tqdm,trange
from collections import defaultdict
import string
import pandas as pd
import numpy as np
import cupy
import cudf
from cudf.core.subword_tokenizer import SubwordTokenizer
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.optim import Adam
from torch.utils.data import TensorDataset, DataLoader
from torch.utils.data.dataset import random_split
from torch.utils.dlpack import from_dlpack

That's extremely confusing when you put a requirements.txt in a folder with a notebook, looks like almost every folder has a requirements.txt or .yml which makes it seem like "build your env using this to run this notebook" right?

Please fix the entire training-tuning-scripts folder if they require fixing but just trying to run the log-parsing-training.ipynb and seeing ModuleNotFoundError over and over and having to construct an env installing one package per error at a time is not nice.

nv-morpheus / Morpheus