Open jrbourbeau opened 4 months ago
Can I disable this and say I'm okay with non-deterministic tokenization?
Hopefully with tokenize.ensure-deterministic=False
Hopefully with
tokenize.ensure-deterministic=False
This didn't seem to do the trick
I believe this should go away with latest main. tokenization is a weird thing and a lot of work went into this recently. Most notably https://github.com/dask/dask/pull/10883 will change this to work I suspect.
The config option is useless since dask-expr hard codes because we rely on this to be actually deterministic
Ah, thanks. I'll try again with the main
branch of dask/dask
.
The config option is useless since dask-expr hard codes because we rely on this to be actually deterministic
Just to clarify, things should still work when using the main
dask
version even though dask-expr
is hardcoded the config option?
Just to clarify, things should still work when using the main dask version even though dask-expr is hardcoded the config option?
dask/dask will respect the config option, dask-expr does not. Was this your question?
We just removed a hack in dask-expr (#822); things will break if you don't upgrade to the latest dask/dask (dask/dask#10883) too
things will break if you don't upgrade to the latest dask/da
We should start being mindful about compatibility and pinning here if we want to keep this as a separate repo/package
I’ve already started pinning the releases to a specific dask/dask version
While writing out a DataFrame to parquet from a Prefect task (not sure if the Prefect part is actually important or not), I got the following error:
A few of things come to mind:
RuntimeError
to be raisedThis might actually be a
dask/dask
issue, but opening up here as I'm currently usingdask-expr
cc @phofl @crusaderky