fix: case sensitive file extension checks and force them to lower cas…

deepset-ai / deepset-cloud-sdk

A Python SDK to interact with deepset Cloud

Apache License 2.0

9 stars 2 forks source link

fix: case sensitive file extension checks and force them to lower cas… #201

Closed FHardow closed 4 months ago

FHardow commented 4 months ago

…e on upload

Related Issues

fixes https://deepset.atlassian.net/browse/DC-1523

Proposed Changes?

Force the file extension to be lower case to check against allowed extensions and on upload. All other checks, like checking for metadata files are left untouched.

How did you test it?

Adjusted unit tests.

Notes for the reviewer

Screenshots (optional)

Checklist

[ ] I have updated the referenced issue with new insights and changes
[ ] If this is a code change, I have added unit tests
[ ] I've used the conventional commit specification for my PR title
[ ] I updated the docstrings
[ ] If this is a code change, I added meaningful logs and prepared Datadog visualizations and alerts

swarmia[bot] commented 4 months ago

✅ Linked to Bug DC-1523 · SDK does not upload files having an uppercase extension

github-actions[bot] commented 4 months ago

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
deepset_cloud_sdk/_service
files_service.py
Project Total

This report was generated by python-coverage-comment-action

mathislucka commented 4 months ago

and checks that only the example.TXT has the correct metadata ?

Are we considering lower and upper case to be different files then?

I think that wouldn't be immediately obvious to me as a user.

FHardow commented 4 months ago

Hey @mathislucka I added a comment in the ticket.

Are we considering lower and upper case to be different files then?

We probably need to. It depends on your file system how case sensitivity is treated. We can't enforce a casing on upload without running into the problem of overwriting data.

FHardow commented 4 months ago

We decided that we will not continue on this fix and document that customers files need to end with lower cased extensions.

mathislucka commented 4 months ago

Could we add an explicit warning to the upload command? I'm afraid that users won't check the documentation.

agnieszka-m commented 4 months ago

The warning could say: Make sure your files have lowercase extensions, for example, my_file.pdf, instead of my_file.PDF. The SDK doesn't upload files with uppercase extensions.