search

BNL-DAQ-LDRD / NeuralCompression

3 stars 2 forks source link

Model data release #18

Closed pphuangyi closed 2 years ago

pphuangyi commented 2 years ago

Major changes in the model_data_release branch of the NeuralCompression repo:

  1. save a pair of pretrained encoder and decoder to the checkpoints folder;
  2. save 8 frames (together with the manifest file needed by the data loader) to the data folder;
  3. add test.py that loads data, does compression and decompression, and saves results to the results folder;
  4. Update the readme file with the test function.
  5. Modified the neuralcompress/utils/tpc_dataloader.py so that we only assert the existence of a manifest file if the partition is required. (So if we only request a few samples for testing, only the manifest for the test partition, test.txt, is needed.)

Please let me know how I can improve it.

YHRen commented 2 years ago

@pphuangyi Thanks a lot. I will review it soon.

YHRen commented 2 years ago

@pphuangyi I'm also thinking to move benchmark script to

pphuangyi commented 2 years ago

Hi Ray, I guess you mean I should move benchmarking code and result out, right? Let me know.

On Mon, Apr 11, 2022 at 12:17 PM YHRen @.***> wrote:

@pphuangyi https://github.com/pphuangyi I'm also thinking to move benchmark script to

— Reply to this email directly, view it on GitHub https://github.com/BNL-DAQ-LDRD/NeuralCompression/pull/18#issuecomment-1095322540, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFMAOSGYHT7GVEWC2BGYLETVERNEHANCNFSM5TDQF2AA . You are receiving this because you were mentioned.Message ID: @.***>

YHRen commented 2 years ago

Hi Ray, I guess you mean I should move benchmarking code and result out, right? Let me know.

@pphuangyi No worries. Sorry. I now remember we want to put other vendors benchmark code here. Let's keep it this way.

Please go through the new readme file, make sure everything looks good. We will open-source this repo after this PR. (hopefully)