COCOS-151 - Add compression/decompression option for CLI/Agent

[SammyOina]

What type of PR is this?

This is a feature that allows on the fly compression and decompression

What does this do?

• New Features

  • Enhanced command-line interface with simplified commands and new options for algorithm and dataset uploads, including checksum validation.
  • Support for in-memory dataset zipping and enhanced context management for decompression flags.

• Bug Fixes

  • Improved error handling for checksum calculations and command executions.

• Documentation

  • Updated CLI documentation to reflect command changes and new features.

• Refactor

  • Streamlined checksum calculations and context management to improve performance and clarity.

Which issue(s) does this PR fix/relate to?

• Resolves #151

Have you included tests for your changes?

No

Did you document any new/modified feature?

Yes

Notes

go run ./cmd/cli/main.go data ./test/manual/data private.pem -d  

# ls /datasets
iris.csv

[SammyOina]

➜  cocos git:(cocos-151) ✗ ./build/cocos-cli data ./test/manual/data private.pem -d

2024/08/16 12:29:55 Uploading dataset CSV: ./test/manual/data

Uploading data...  100% [====================================================>] 

{"time":"2024-08-16T12:29:55.989831624+03:00","level":"ERROR","msg":"Failed to send Data"}

2024/08/16 12:29:55 Error uploading dataset: rpc error: code = Unknown desc = malformed data, filename does not match manifest

How do you handle and verify file name in this structure since it does not work

Use the base name of the directory, check code to see how directories are handled

[SammyOina]

This scheme works with the test computations server but not with prism, please check where the issue lies.

Which scheme, and what does not work?

[SammyOina]

➜  cocos git:(cocos-151) ✗ ./build/cocos-cli data ./test/manual/data private.pem -d
2024/08/16 12:29:55 Uploading dataset CSV: ./test/manual/data
Uploading data...  100% [====================================================>] 
{"time":"2024-08-16T12:29:55.989831624+03:00","level":"ERROR","msg":"Failed to send Data"}
2024/08/16 12:29:55 Error uploading dataset: rpc error: code = Unknown desc = malformed data, filename does not match manifest

How do you handle and verify file name in this structure since it does not work

base name here will be data if you can refer here https://github.com/SammyOina/cocos-ai/blob/1cde120d705e73cdc7fdb7993011804ba1da8c65/cli/datasets.go#L53