search

dwavesystems / dwave-cloud-client

A minimal implementation of the REST interface used to communicate with D-Wave Solver API (SAPI) servers.
https://docs.ocean.dwavesys.com/projects/cloud-client/en/stable/
Apache License 2.0
58 stars 39 forks source link

Compress (in transport) BQM/CQM/DQM data during multipart upload #532

Open randomir opened 2 years ago

randomir commented 2 years ago

Since serialized BQM et al. data is not compressed in dimod anymore, look into benefits of compressing in transit.

arcondello commented 2 years ago

DQM.to_file() supportes compress, it would be pretty easy to add it to CQM/BQM if desired. Would need to measure the performance

randomir commented 2 years ago

That would be nice. If all supported compress flag, we could use it by default from the client.

arcondello commented 2 years ago

Sure, made an issue https://github.com/dwavesystems/dimod/issues/1235. I do wonder what the memory/time cost of compressing the overall model rather than the individual parts in multi part upload. Obviously compressing the overall object should lead to better compression, but I would also expect it to be slower.

randomir commented 2 years ago

Benefits also heavily depend on network speed. On (very) fast networks, using anything except the simplest/fastest stream compression is probably not worth it. But we upload in parallel, so that tips the balance back... Hard to tell without some benchmarks. "Optimal" upload parameters will definitely be informed by benchmarks.