Before this PR wrong location was displayed for created deployment directory, this happened because after creating a deployment directory, it is moved to a different location at sparsify level (but there were no logs for this step), this was a source of confusion for the user
Current PR fixes that by adding a logger to sparsify runner, at info level and logging the directory move displaying correct information to a potential user
Before this PR (shows deployment_dir under training_artifacts which is incorrect):
2023-07-06 21:33:04 sparseml.transformers.export WARNING Optional file merges.txt not found in /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_32_30/training_artifacts/. Skipping copying to deployment folder.
WARNING:sparseml.transformers.export:Optional file merges.txt not found in /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_32_30/training_artifacts/. Skipping copying to deployment folder.
2023-07-06 21:33:04 sparseml.transformers.export INFO Created deployment folder at /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_32_30/training_artifacts/deployment with files: ['model.onnx', 'config.json', 'tokenizer_config.json', 'special_tokens_map.json', 'tokenizer.json']
INFO:sparseml.transformers.export:Created deployment folder at /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_32_30/training_artifacts/deployment with files: ['model.onnx', 'config.json', 'tokenizer_config.json', 'special_tokens_map.json', 'tokenizer.json']
After this PR:
INFO:sparseml.transformers.export:Created deployment folder at /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_33_57/training_artifacts/deployment with files: ['model.onnx', 'config.json', 'tokenizer_config.json', 'special_tokens_map.json', 'tokenizer.json']
train_directory='/home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_33_57/training_artifacts', deploy_directory='/home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_33_57/deployment'
INFO:sparsify.auto.tasks.runner:Moving /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_33_57/training_artifacts/deployment to /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_33_57/deployment
INFO:sparsify.auto.tasks.runner:Deleting /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_33_57/training_artifacts/deployment
INFO:sparsify.auto.tasks.runner:Deployment directory moved to /home/rahul/projects/sparse_transfer_token_classification_2023_07_06_21_33_57/deployment
The move is now logged accurately and correct deployment_dir location is displayed
Note: The test command was run locally with a short circuited run (setting max_steps to 10)
To see the specific tasks where the Asana app for GitHub is being used, see below:
Before this PR wrong location was displayed for created deployment directory, this happened because after creating a deployment directory, it is moved to a different location at sparsify level (but there were no logs for this step), this was a source of confusion for the user
Current PR fixes that by adding a logger to sparsify runner, at info level and logging the directory move displaying correct information to a potential user
Test Command:
Before this PR (shows deployment_dir under training_artifacts which is incorrect):
After this PR:
The move is now logged accurately and correct deployment_dir location is displayed Note: The test command was run locally with a short circuited run (setting max_steps to 10)