lancedb / lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
https://lancedb.github.io/lance/
Apache License 2.0
3.99k stars 228 forks source link

Commit conflict error in Python just gives module not found #2530

Open wjones127 opened 5 months ago

wjones127 commented 5 months ago

When a commit conflict happens, sometimes users just get this message instead:

Error importing from pylance AttributeError: module 'lance' has no attribute 'commit', src/dataset/commit.rs:41:27

This is failing the import at

https://github.com/lancedb/lance/blob/06a6a09561493b550954a3ed99cdf2dc551bd748/python/src/dataset/commit.rs#L27-L30

wjones127 commented 5 months ago

Two things:

wjones127 commented 5 months ago

We have a whole test here:

https://github.com/lancedb/lance/blob/63227f4e2755cddbc27b6046e03a0df0f06869c1/python/python/tests/test_dataset.py#L1492-L1502

I'm not sure then why this would happen. It must be on AWS lambda then somehow. Maybe I can write a simple script to repro and run on there.

Yitzchak-SB commented 5 months ago

I believe the Exception was raised inside the yield:

 @contextlib.contextmanager 
 def commit_lock(_version: int): 
     try: 
         raise Exception("hello world!") 
     finally: 
         pass

 with pytest.raises(Exception, match="hello world!"): 
     lance.write_dataset( 
         pa.table({"a": range(100)}), tmp_path / "test2", commit_lock=commit_lock 
     )