search

pinecone-io / pinecone-text

Pinecone text client library
Other
52 stars 29 forks source link

Downloading BM25 Encoders using Pinecone-Text in Celery is failing #82

Open saharaquant opened 3 weeks ago

saharaquant commented 3 weeks ago

Is this a new bug?

Current Behavior

When trying to init a BM25 encoder bm25_encoder = BM25Encoder().default() during a celery task - it's throwing an error 'LoggingProxy' object has no attribute 'fileno'

This issue is happening because of wget trying to print a bar, and there's no way to control it from the outside. https://github.com/pinecone-io/pinecone-text/blob/main/pinecone_text/sparse/bm25_encoder.py#L261

Expected Behavior

Provide a way to control if it should display a bar or not when downloading the encoder

Steps To Reproduce

Run the simplest celery task trying to execute bm25_encoder = BM25Encoder().default()

Relevant log output

bm25_encoder = BM25Encoder().default()\n File \"\/usr\/local\/lib\/python3.10\/site-packages\/pinecone_text\/sparse\/bm25_encoder.py\", line 261, in default\n wget.download(url, str(tmp_path))\n File \"\/usr\/local\/lib\/python3.10\/site-packages\/wget.py\", line 526, in download\n (tmpfile, headers) = ulib.urlretrieve(binurl, tmpfile, callback)\n File \"\/usr\/local\/lib\/python3.10\/urllib\/request.py\", line 267, in urlretrieve\n reporthook(blocknum, bs, size)\n File \"\/usr\/local\/lib\/python3.10\/site-packages\/wget.py\", line 513, in callback_charged\n callback_progress(blocks, block_size, total_size, bar_function=bar)\n File \"\/usr\/local\/lib\/python3.10\/site-packages\/wget.py\", line 461, in callback_progress\n width = min(100, get_console_width())\n File \"\/usr\/local\/lib\/python3.10\/site-packages\/wget.py\", line 337, in get_console_width\n ioctl(sys.stdout.fileno(), TIOCGWINSZ, winsize)\nAttributeError: 'LoggingProxy' object has no attribute 'fileno'\n

Environment

- **OS**: Linux
- **Language version**: 3.10.4
- **Pinecone client version**: 0.5.3, but also on the latest

Additional Context

https://github.com/pinecone-io/pinecone-text/blob/main/pinecone_text/sparse/bm25_encoder.py#L261

saharaquant commented 2 weeks ago

Anyone?