Closed ceyhunkerti closed 1 month ago
Currently reading http ndjson with lazy option is not supported 👇🏼
iex(2)> frame = Explorer.DataFrame.from_ndjson("http://127.0.0.1:8080/demo.ndjson", lazy: true) ** (FunctionClauseError) no function clause matching in Explorer.PolarsBackend.LazyFrame.from_ndjson/3 The following arguments were given to Explorer.PolarsBackend.LazyFrame.from_ndjson/3: # 1 %FSS.HTTP.Entry{ url: "http://127.0.0.1:8080/demo.ndjson", config: %FSS.HTTP.Config{headers: []} } # 2 1000 # 3 1000 Attempted function clauses (showing 2 out of 2): def from_ndjson(%FSS.S3.Entry{}, _, _) def from_ndjson(%FSS.Local.Entry{} = entry, infer_schema_length, batch_size) (explorer 0.10.0-dev) lib/explorer/polars_backend/lazy_frame.ex:230: Explorer.PolarsBackend.LazyFrame.from_ndjson/3 iex:2: (file)
demo.ndjson
{"a":1} {"b":2}
http-server .
iex(3)> frame = Explorer.DataFrame.from_ndjson("http://127.0.0.1:8080/demo.ndjson", lazy: true) {:ok, #Explorer.DataFrame< LazyPolars[??? x 2] a s64 [1, nil] b s64 [nil, 2] >}
demo.ndjson.tar.gz
@ceyhunkerti thank you!
Currently reading http ndjson with lazy option is not supported 👇🏼
Manual Test
demo.ndjson
http-server .
on the path containingdemo.ndjson
demo.ndjson.tar.gz