search

Yale-LILY / NLP4Code

Repository for the NLP4Code project at the LILY lab.
Apache License 2.0
5 stars 1 forks source link

troyfeng116/tests #45

Open troyfeng116 opened 1 year ago

niansong1996 commented 1 year ago

@troyfeng116 LMK if you want my input on anything

niansong1996 commented 1 year ago

For integration test, one suggestion is that instead of doing what I did in tests/test_inference_pipeline.py, you should follow this doc and run it with Lightning API instead

troyfeng116 commented 1 year ago

For integration test, one suggestion is that instead of doing what I did in tests/test_inference_pipeline.py, you should follow this doc and run it with Lightning API instead

Got it, currently working on models (something similar to what I did for datasets if you have time to take a look at that).

Would using Lightning API just be to avoid using os.system?

niansong1996 commented 1 year ago

Would using Lightning API just be to avoid using os.system?

Yes, but it seems like this function is not added for the version of the lightning we are using (1.7.4). So let's stall this.

niansong1996 commented 1 year ago

But I found another issue: I was trying to use GPUs for the tests but got error like this you've requested GPU [0,1] but only got [0]. I suspect that when you start the runner, there is some env var that sets CUDA_VISIBLE_DEVICES=0 somewhere. Can you check this out?

troyfeng116 commented 1 year ago

But I found another issue: I was trying to use GPUs for the tests but got error like this you've requested GPU [0,1] but only got [0]. I suspect that when you start the runner, there is some env var that sets CUDA_VISIBLE_DEVICES=0 somewhere. Can you check this out?

Should have just fixed, left an env vars from SummerTime I think

niansong1996 commented 1 year ago

I just merged #40 into main, so please merge from main and fix the conflicts. But definitely feel free to ping me if you need me on anything.

troyfeng116 commented 1 year ago

@niansong1996 Running into some problem with spider_reader, seems like example_to_demonstration_sql was deprecated so promptify_function function pointer can't be resolved?

niansong1996 commented 1 year ago

Ignore the non-few-shot datasets for now and add an issue as a TODO instead.

troyfeng116 commented 1 year ago

@niansong1996 done expanding all tests, takes about 9-10 minutes to run at the moment. Let me know if it look ok for now, ready to merge anytime