Closed nebulae closed 1 year ago
replaced ndjson with jsonlines replaced patool with rarfile and py7zr
to note - jsonlines and rarfile are both python 3.6+, whereas the replaced patool and ndjson both support python 2.7.
import eta.core.serial as etas import fiftyone as fo import fiftyone.zoo as foz ndjson_path = "/tmp/samples.ndjson" dataset = foz.load_zoo_dataset("cifar10", split="test", max_samples=10) objs = list(s.to_dict() for s in dataset) etas.write_ndjson(objs, ndjson_path) assert etas.read_ndjson(ndjson_path) == objs with open(ndjson_path, "r") as f: assert etas.load_ndjson(f.read()) == objs with open(ndjson_path, "rb") as f: assert etas.load_ndjson(f.read()) == objs
import eta.core.utils as etau import eta.core.web as etaw url = "http://serre-lab.clps.brown.edu/wp-content/uploads/2013/10/test_train_splits.rar" rar_path = "/tmp/test_train_splits.rar" etaw.download_file(url, path=rar_path) etau.extract_archive(rar_path)
ls -lah /tmp/
changelog
replaced ndjson with jsonlines replaced patool with rarfile and py7zr
to note - jsonlines and rarfile are both python 3.6+, whereas the replaced patool and ndjson both support python 2.7.
ndjson test
RAR test