search

tensorflow / models

Models and examples built with TensorFlow
Other
77.04k stars 45.77k forks source link

pQRNN #9277

Closed loretoparisi closed 2 years ago

loretoparisi commented 4 years ago

Prerequisites

Please answer the following question for yourself before submitting an issue.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/tree/master/research/sequence_projection

2. Describe the feature you request

Release the PRADO generation model for pQRNN.

3. Additional context

Inference on large or medium size documents currently needs high end machines to handle in memory vectors for BERT like models. Only recent advancements in tokenization (SentencePiece, etc.) made it possible to speed up inference time, but not the model inference time itself.

4. Are you willing to contribute it? (Yes or No)

Yes

yugaljain1999 commented 4 years ago

@loretoparisi @saikumarchalla I am willing to contribute in this project and I have huge amount of labelled data as well to test this model and improve this model further if needed by collabing with other collaborators of tensorflow NLP models.Please assigned this issue to me. Regards!

thunderfyc commented 4 years ago

@loretoparisi @yugaljain1999 Thanks for your interest on this model. Can you clarify a bit more on your feature request? My understanding is you are requesting to open source pQRNN model.

Welcome to create your own model based what we open sourced here, and use it anywhere (don't forget to tell us the result and impact if it's sharable)

We are working on TF2 + keras version of PRADO and pQRNN, which may cause significant code structure change on current codebase.

If you'd like to make your model merged to this code base to make your contribution more widely visible, be sure to talk with before creating the PR

loretoparisi commented 4 years ago

@thunderfyc Yes, my specific request is about to open source the pQRNN model as you just did for PRADO (thanks for that!).

yugaljain1999 commented 4 years ago

@thunderfyc I agree with @loretoparisi , my request is also about to open source pQRNN as it is next level version of PRADO and nicely worked on various datasets..

yugaljain1999 commented 4 years ago

@loretoparisi How did u run PRADO model on your system as I am running on google colab and getting error related to bazel which I tried to install using build in commands in colab but even then it didn't run as it should. Can u please help me to resolve bazel error? Thanks!

loretoparisi commented 4 years ago

@yugaljain1999 let me check it out.

yugaljain1999 commented 4 years ago

@loretoparisi Have u find any way to run PRADO model? I am facing an issue in importing one module because it actually doesn't exist in original files but in runner.py it's written. @loretoparisi @thunderfyc I got "no module found named tf_ops "error while running prado_model.py. Error is in the line number 25 from tf_ops import sequence_string_projection_op as ssp I couldn't get any info regarding tf_ops ,from where it came out as tf_ops.py is not any file which exist in directory named "prado". Do you how to import tf_ops to solve this error? It would be great help.

image Thanks.

prrao87 commented 4 years ago

Hi @thunderfyc I just wanted to check if there was any progress on the plans to open source the pQRNN (TF2 + keras) model? Was just curious about the rough timeline, if you had one. Thanks!

prabhukaliamoorthi commented 4 years ago

We are open sourcing PRADO (TF2 + keras) in the next couple of weeks. pQRNN is likely six months away.

Gorluxor commented 4 years ago

@prabhukaliamoorthi Thanks on the general timeline! I Would love to try the model when its in TF2 + keras, if possible with custom dataset (non english).

@yugaljain1999 I succeeded in running it in google colab with bazel instructions. You can check out the first part of notebook I used to recreate the runtime environment: https://colab.research.google.com/drive/1XXwlZyYPnmVFtmwqWDrqZg9TFfgT02Ya?usp=sharing

yugaljain1999 commented 4 years ago

@Gorluxor Thankyou so much for your notebook, it would really help me a lot. Can I run this notebook on GPU as well ? Thanks!

Gorluxor commented 4 years ago

@yugaljain1999 When I ran it in a gpu environment, it said it isn't utilized, so I believe it's not useful at all. On the other hand when you run the training, it has references to TPU (so either TPU or just cpu). Overall I find the general process quite strange compared to normal training tf/keras models (at least for me).

yugaljain1999 commented 4 years ago

@Gorluxor you are right I also find it strange when I ran this on GPU and it was throwing an error when running on GPU during training. Can I save weights after training this model on TPU as I didn't run any model on TPU till now and load those weights when using for deployment or production? Thanks!

Gorluxor commented 4 years ago

@yugaljain1999 Wouldn't know to help you with that, as I don't know much about bazel or how is this running in the background. I wanted to try it myself yesterday, so that is why I figured out the environment. Still would not know how to change it to use my dataset for NER problem.

yugaljain1999 commented 4 years ago

@Gorluxor no problem, I will myself too to load weights of this model trained on TPU and try to fit my data. Thanks a lot.

Gorluxor commented 4 years ago

@yugaljain1999 On which dataset are you trying to train it on (saw that you said you have a huge amount of labeled data)? And what are you trying to predict? Plus did you manage to easily change the dataset to fit the model?

prabhukaliamoorthi commented 3 years ago

The keras+TF2 version is out.

prrao87 commented 3 years ago

@prabhukaliamoorthi That's great - could you please post the link to the model/code here if possible? Thanks!

Gorluxor commented 3 years ago

@prrao87 I assume, this is the revised code https://github.com/tensorflow/models/tree/master/research/seq_flow_lite, just that it changed name under models

jasonw247 commented 3 years ago

Is there any update on the pQRNN model release timeline? Thanks!

yugaljain1999 commented 3 years ago

Yes, it's released but unofficially. If you want I can share repo. Thanks

On Thu, Mar 4, 2021, 4:15 AM jasonw247 notifications@github.com wrote:

Is there any update on the pQRNN model release timeline? Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/models/issues/9277#issuecomment-790126312, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHY3MAC42TDDWRCV5GQBOETTB23ZPANCNFSM4RVALWWQ .

jasonw247 commented 3 years ago

@yugaljain1999 that would be great, thanks!

yugaljain1999 commented 3 years ago

Here is the repo - https://github.com/ChenghaoMou/pytorch-pQRNN

On Thu, 4 Mar, 2021, 8:25 am jasonw247, notifications@github.com wrote:

@yugaljain1999 https://github.com/yugaljain1999 that would be great, thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/models/issues/9277#issuecomment-790247949, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHY3MAEQ7HM4WCEQVVT24PTTB3ZA7ANCNFSM4RVALWWQ .

AetherPrior commented 3 years ago

Hi everyone! Has the official model (trained model and code) for PQRNN been released yet? Or are there any plans for doing so?
There are unofficial implementations, but are there any plans for releasing the official one? Thank you!

karunreddy30 commented 3 years ago

Hi everyone! Has the official model (trained model and code) for PQRNN been released yet? Or is there any plans for doing so? There are unofficial implementations, but is there any plans for releasing the official one? Thank you!

We're working towards releasing them in Q4.

AetherPrior commented 2 years ago

Can we close this issue? An opensourced version of PQRNN has been merged (#10475 )

loretoparisi commented 2 years ago

definitely!