Closed randomgambit closed 2 years ago
summoning the masters @LysandreJik @sgugger @Rocketknight1 💯
Hey! There are a couple of issues here. The first is that we're trying to move away from TFTrainer towards Keras - there'll be a new version of that notebook coming soon, like I promised!
In the meantime, your approach should work, though. The error you're getting is because lm_datasets
is actually a DatasetDict
containing both the train and validation set, so everything downstream gets confused. You probably want to swap out lm_datasets
for lm_datasets['train']
in that call to TFTrainer
. However, like I said, we're trying to deprecate TFTrainer, so I'm trying to avoid doing any more bugfixing for it. I'm working on getting the new examples in ASAP!
Thanks @Rocketknight1 ! Actually I was getting the same error even when I was using a dataset
that only contains one set of data. But you are absolutely right: there is no need to fix something that is going to be deprecated soon. Happy to help if you need anything! Thanks!
The good news is I'm moving to working on those TF notebooks right now, so hopefully I'll have a proper example to show you soon. However, the official launch of the new notebooks might depend on the PR at https://github.com/huggingface/datasets/pull/2731 being accepted and making it to release, since I'm planning to use that new method in a lot of them.
Still, I'll make sure to ping you as soon as I have a LM example ready - just be aware that you might have to install a pre-release version of datasets
to get it to work!
got it. happy to try out the beta version of them at my risk and peril ;-)
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
Same question at year 2023 for https://github.com/huggingface/notebooks/blob/main/examples/language_modeling_from_scratch.ipynb
Hello there!
I tried to use the language-modeling-from-scratch notebook https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/language_modeling_from_scratch.ipynb#scrollTo=JEA1ju653l-p
More specifically, I need to run it by using
tensorflow
. The simple strategy of using theTF
versions of thehuggingface
functions everything seems to work correctly until I reach thetrainer
step and then I get a mysterious cardinality issue.This looks like a bug... Can you please have a look at the code below?
What do you think? Thanks!