Error in Python 3.9.16

[konkhra]

Traceback (most recent call last): File "/root/wetransfer/transferwee.py", line 650, in <module> print(upload(args.files, args.n, args.m, args.f, args.t)) File "/root/wetransfer/transferwee.py", line 551, in upload _storm_preflight(transfer["storm_upload_token"], files) File "/root/wetransfer/transferwee.py", line 329, in _storm_preflight return r.json() File "/usr/lib/python3.9/site-packages/requests/models.py", line 891, in json return complexjson.loads( File "/usr/lib64/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/lib64/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[iamleot]

I could not reproduce that, sorry.

I have tried to run ./check.sh and everything worked fine.

Please share how to reproduce that with the exact commands.

Thanks!

[konkhra]

I am running with this command line python3 transferwee.py upload -m 'Just a message ...' transferwee.py

Here is the package list installed on my system:

`Package Version

---

Babel 2.9.1 beautifulsoup4 4.12.2 chardet 4.0.0 click 8.1.3 cloud-init 22.1 configobj 5.0.6 cssselect 1.2.0 dbus-python 1.2.18 distro 1.5.0 feedfinder2 0.0.4 feedparser 6.0.10 filelock 3.12.0 gpg 1.15.1 idna 2.10 iotop 0.6 jieba3k 0.35.1 Jinja2 2.11.3 joblib 1.2.0 jsonpatch 1.21 jsonpointer 2.0 libcomps 0.1.18 lxml 4.9.2 MarkupSafe 1.1.1 netifaces 0.10.6 newspaper3k 0.2.8 nltk 3.8.1 oauthlib 3.1.1 pcp 5.0 Pillow 9.5.0 pip 23.1.2 prettytable 0.7.2 pyparsing 2.4.7 pyserial 3.4 PySocks 1.7.1 python-dateutil 2.8.1 pytz 2021.1 PyYAML 5.4.1 regex 2023.5.5 requests 2.25.1 requests-file 1.5.1 rpm 4.16.1.3 selinux 3.5 sepolicy 3.5 setools 4.4.1 setuptools 53.0.0 sgmllib3k 1.0.0 six 1.15.0 soupsieve 2.4.1 systemd-python 234 tinysegmenter 0.3 tldextract 3.4.4 tqdm 4.65.0 urllib3 1.26.5`

[iamleot]

Here doing that here:

$ python3 transferwee.py upload -v -m 'Just a message ...' transferwee.py
Checking that all files exists
Checking for no duplicate filenames
Preparing to upload
Could not find any csrf-token
Get transfer id d25678edbb6471d68e072047a4ac8df120230604154732
Doing preflight storm
Preparing storm block upload
Uploading file transferwee.py
Finalizing storm batch upload
Request against https://storm-eu-west-1.wetransfer.net/api/v2/batch returned 425, retrying in 1 seconds
Finalizing upload with transfer id d25678edbb6471d68e072047a4ac8df120230604154732
https://we.tl/t-YDj9hwOuz8

(using requests-2.28.1 and python-3.10.11 but I don't think that neither of requests and Python version are relevant here.)

If you can reliably reproduce that probably trying to add some breakpoint() and printing the corresponding HTTP response can shed some light.

Also using a possible HTTP proxy (e.g. mitmproxy or proxify or similar) that intercept all HTTP(S) requests and logs them can help to figure out what is going wrong.

[cfrans]

I'm getting a similar error using the command python3 transferwee.py upload -v -m 'Just a message ...' transferwee.py, already tried to run in Windows, Linux and using Docker. Also tried using requests-2.28.1 and python-3.10.11 but the same error persists:

Checking that all files exists
Checking for no duplicate filenames
Preparing to upload
Could not find any csrf-token
Get transfer id a422671df40299133c057773529f227120230607170001
Doing preflight storm
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 960, in json
    return complexjson.loads(self.content.decode(encoding), **kwargs)
  File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/./transferwee.py", line 650, in <module>
    print(upload(args.files, args.n, args.m, args.f, args.t))
  File "/app/./transferwee.py", line 551, in upload
    _storm_preflight(transfer["storm_upload_token"], files)
  File "/app/./transferwee.py", line 329, in _storm_preflight
    return r.json()
  File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 968, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[konkhra]

Could be that some IPs are filtered by wetransfer and the output is not as expected but I do not know how to debug that with the script. My programming knowledge is very limited.

[iamleot]

Could be that some IPs are filtered by wetransfer and the output is not as expected but I do not know how to debug that with the script. My programming knowledge is very limited.

Yeah, for sure there is something interesting going on...

I think that a possible problem is that there should be several sources of the "storm" thingie (probably related to a subset of AWS regions used by WeTransfer under the hood) and at the moment only the eu-west-1 is used. (I've tried to look at the JS files fetched and tried to grep around for several AWS regions but I was not able to find anything so I still have not figured out how that region is picked up)

Maybe some assertion and/or more debugging logs could be added to hopefully shed some more light.

[iamleot]

I've tried to investigate more and I think that PR #53 can hopefully fix/help it!

I believe that when doing the first request in the token - that is a JWT - we also get the Storm URLs for which we are authorized. In my case - being near eu-west-1 - everything always went fine. But if for any reason a different region was chosen I think that we get 401 errors because we should actually authenticate against non-eu-west-1 URLs.

[iamleot]

Can you please retry with latest commit (commit 286d5ba) and let me know if that's fine?

Thank you!

[konkhra]

Great! I confirm it is working now. Tested on VPS located in US east coast which before gave me the error.

[iamleot]

Yay, cool! Thank you for the quick feedback!

[iamleot]

Closing, commit 286d5ba fixed it.