y500 / libtorrent

Automatically exported from code.google.com/p/libtorrent
0 stars 0 forks source link

Windows binaries for python 2.7 doesn't work #699

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
What steps will reproduce the problem?

I use this code

import libtorrent as lt
import time

ses = lt.session()
ses.listen_on(6881, 6891)

torrent_fname = ur"<censored>"

e = lt.bdecode(open(torrent_fname, 'rb').read())
info = lt.torrent_info(e)

params = {
    'save_path': './',
    'storage_mode': lt.storage_mode_t.storage_mode_sparse,
    'ti': info,
}
h = ses.add_torrent(params)

while (not h.is_seed()):
        s = h.status()

        print '%.2f%% complete (down: %.1f kb/s up: %.1f kB/s peers: %d seeds: %d) %s' % \
                (s.progress * 100, s.download_rate / 1000, s.upload_rate / 1000,
                s.num_peers, s.num_seeds, s.state )

        time.sleep(1)

What is the expected output? What do you see instead?

Output (torrent 1):

0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) 
checking_resume_data
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 2 seeds: 1) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 2 seeds: 1) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 2 seeds: 1) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 3 seeds: 3) downloading
0.00% complete (down: 2.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 4.0 kb/s up: 1.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 3.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 2.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 2.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 1.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 1.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 1.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading

Torrent 2.

0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) 
checking_resume_data
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 1 seeds: 1) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 2 seeds: 2) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 2 seeds: 2) downloading
0.00% complete (down: 7.0 kb/s up: 0.0 kB/s peers: 2 seeds: 2) downloading
0.00% complete (down: 17.0 kb/s up: 0.0 kB/s peers: 2 seeds: 2) downloading
0.00% complete (down: 39.0 kb/s up: 1.0 kB/s peers: 3 seeds: 3) downloading
0.00% complete (down: 75.0 kb/s up: 2.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 80.0 kb/s up: 3.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 64.0 kb/s up: 2.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 51.0 kb/s up: 1.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 41.0 kb/s up: 1.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 33.0 kb/s up: 1.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 26.0 kb/s up: 1.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 21.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 16.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 13.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 10.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 8.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 6.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 5.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 4.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 3.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 2.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 2.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 1.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 1.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 1.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading
0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0 seeds: 0) downloading

What version of the product are you using? On what operating system?

python 2.7.5 on windows. Libtorrent downloaded from 
https://libtorrent.googlecode.com/files/python-libtorrent-0.16.10.win32.msi

Please provide any additional information below.

uTorrent downloads same torrents normally. 2.6 binaries 
https://libtorrent.googlecode.com/files/python-libtorrent-0.15.10.win32-py2.6.ms
i work too. This seems to be reported a year ago 
https://code.google.com/p/libtorrent/issues/detail?id=481#c4

Original issue reported on code.google.com by xpeh.o...@googlemail.com on 22 Nov 2014 at 3:11

GoogleCodeExporter commented 8 years ago
could you enable all alerts (ses.set_alert_mask()) and print all the alerts you 
get? This report does not contain enough information to pin point what's going 
on. My first guess would be that the peers you get may be sending corrupt data 
and they get banned.

have you tried other torrents?

Original comment by arvid.no...@gmail.com on 22 Nov 2014 at 4:28

GoogleCodeExporter commented 8 years ago
[deleted comment]
GoogleCodeExporter commented 8 years ago
@arvid as I mentioned, I tried 2 torrents which work wonderful with uT and 
previous version of libtorrent.

What code should I add?

Original comment by xpeh.o...@googlemail.com on 22 Nov 2014 at 6:59

GoogleCodeExporter commented 8 years ago
--- temp.py 2014-11-22 01:12:00.000000000 -0800
+++ temp2.py    2014-11-22 01:30:39.000000000 -0800
@@ -3,6 +3,7 @@

 ses = lt.session()
 ses.listen_on(6881, 6891)
+ses.set_alert_mask(0xffffffff)

 torrent_fname = ur"<censored>"

@@ -23,5 +24,9 @@
                 (s.progress * 100, s.download_rate / 1000, s.upload_rate / 1000,
                 s.num_peers, s.num_seeds, s.state )

+        alert_queue = ses.pop_alerts()
+        for a in alert_queue:
+                print(a.message())
+
         time.sleep(1)

Original comment by arvid.no...@gmail.com on 22 Nov 2014 at 9:30

GoogleCodeExporter commented 8 years ago
http://pastebin.com/tAVDwsrW

There is no ses.pop_alerts()

        for a in iter(session.pop_alert, None):
            print >>f, a.message()

Lets do it from the other side. Can you download any torrent with given 
configuration?

Original comment by xpeh.o...@googlemail.com on 22 Nov 2014 at 10:35

GoogleCodeExporter commented 8 years ago
I don't have a windows machine easily accessible right now. Do you get any 
alerts? any indications of what error you might have?

one guess would be that you use a forward-slash in your save-path on windows, 
instead of backslash.

Original comment by arvid.no...@gmail.com on 24 Nov 2014 at 9:38

GoogleCodeExporter commented 8 years ago
See the link I gave. AFAIK windows is tolerant to slashes.

I think you have to get windows sooner or later if you want to fix this problem.

Original comment by xpeh.o...@googlemail.com on 24 Nov 2014 at 10:32

GoogleCodeExporter commented 8 years ago
> AFAIK windows is tolerant to slashes.

It is sometimes, but not in UNC paths. See 
http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.as
px#maxpath

specifically: "Note  File I/O functions in the Windows API convert "/" to "\" 
as part of converting the name to an NT-style name, except when using the 
"\\?\" prefix as detailed in the following sections."

Original comment by arvid.no...@gmail.com on 24 Nov 2014 at 7:25

GoogleCodeExporter commented 8 years ago
Do you plan to fix it?

Original comment by xpeh.o...@googlemail.com on 3 Dec 2014 at 10:27

GoogleCodeExporter commented 8 years ago
Do you mean accepting unix paths on windows?

I'm not sure of a safe and reliable way to do that. On windows (ntfs at least) 
a filename is allowed to contain a forward slash. Should libtorrent always 
assume that a forward slash in the path is a directory separator?

did you confirm that this is what is causing the problem you're experiencing?

Original comment by arvid.no...@gmail.com on 4 Dec 2014 at 1:35

GoogleCodeExporter commented 8 years ago
No, I mean issue with 2.7 version not downloading. I have no glue why it's 
happening. I uploaded the log, but it doesn't actually say much for my opinion.

Original comment by xpeh.o...@googlemail.com on 5 Dec 2014 at 5:51

GoogleCodeExporter commented 8 years ago
Could you post verbose logs from a test run that fails?

Original comment by arvid.no...@gmail.com on 9 Dec 2014 at 3:18

GoogleCodeExporter commented 8 years ago
Which test run? I don't know what you mean. Can you tell me how to produce them?

Original comment by xpeh.o...@googlemail.com on 11 Dec 2014 at 5:13

GoogleCodeExporter commented 8 years ago
the test run where you fail to download anything.

you get logs by building with: "b2 logging=verbose"

Original comment by arvid.no...@gmail.com on 11 Dec 2014 at 8:22

GoogleCodeExporter commented 8 years ago
I haven't compiled it, I downloaded a binary builds from links I provided. Why 
don't you want to test it by yourself? I can give you a torrent link if you 
cannot find one but every one should be fine.

Original comment by xpeh.o...@googlemail.com on 12 Dec 2014 at 3:44

GoogleCodeExporter commented 8 years ago
The torrents I test with work fine. sure, can you send me one that reproduces 
the issue to arvid@libtorrent.org?

Original comment by arvid.no...@gmail.com on 12 Dec 2014 at 3:50

GoogleCodeExporter commented 8 years ago
[deleted comment]
GoogleCodeExporter commented 8 years ago
Here some filemon logs for 2.7 version:

1467    11:23:44    python.exe:3236 OPEN    C:\path_to_program\.\dir_inside_torrent\file.
mdf PATH NOT FOUND  Options: OpenIf Sequential  Access: All 
1468    11:23:44    python.exe:3236 OPEN    C:\path_to_program\ SUCCESS Options: Open 
Directory  Access: All  
1469    11:23:44    python.exe:3236 DIRECTORY   C:\path_to_program\ SUCCESS FileBothDire
ctoryInformation: dir_inside_torrent    
1470    11:23:44    python.exe:3236 CLOSE   C:\path_to_program\ SUCCESS     
1471    11:23:44    python.exe:3236 OPEN    C:\path_to_program\.\dir_inside_torrent\file.
mdf PATH NOT FOUND  Options: OpenIf Sequential  Access: All 
1472    11:23:44    python.exe:3236 OPEN    C:\path_to_program\.\dir_inside_torrent\file.
mdf PATH NOT FOUND  Options: Open Sequential  Access: All   
1473    11:23:44    python.exe:3236 OPEN    C:\path_to_program\.\dir_inside_torrent\file.
mdf PATH NOT FOUND  Options: OpenIf Sequential  Access: All 
1474    11:23:44    python.exe:3236 OPEN    C:\path_to_program\ SUCCESS Options: Open 
Directory  Access: All  

2.6 version doesn't try to open path with \.\ inside. Also, as you can see in 
the start post, it prints "checking_resume_data" only once, regardless of 
amount of data on disk while 2.6 version indeed rehashes files and that line is 
printed many times. So the reason is probably the same as here 
http://stackoverflow.com/a/6930555

But why is there no logs about failing to open path? 

https://www.archlinux.org/releng/releases/2014.12.01/torrent/ here is the 
example torrent.

Original comment by xpeh.o...@googlemail.com on 17 Dec 2014 at 9:40

GoogleCodeExporter commented 8 years ago
I believe this is fixed by [10620] in RC_1_0

Original comment by arvid.no...@gmail.com on 18 Dec 2014 at 7:37

GoogleCodeExporter commented 8 years ago
Yea, any windows binaries?

Are file access errors now properly logged?

Original comment by xpeh.o...@googlemail.com on 18 Dec 2014 at 7:56

GoogleCodeExporter commented 8 years ago
Why it's "fixed" if there is no windows binaries for that version?

Original comment by xpeh.o...@googlemail.com on 19 Dec 2014 at 12:56

GoogleCodeExporter commented 8 years ago
Would you please answer?

Original comment by xpeh.o...@googlemail.com on 22 Dec 2014 at 2:31

GoogleCodeExporter commented 8 years ago
it's "fixed" because the bug has been corrected in the head of all relevant 
branches.

Original comment by arvid.no...@gmail.com on 22 Dec 2014 at 3:52

GoogleCodeExporter commented 8 years ago
The issue was named "Windows binaries for python 2.7 doesn't work". Can you 
provide working binaries for python 2.7?

Original comment by xpeh.o...@googlemail.com on 24 Dec 2014 at 1:24

GoogleCodeExporter commented 8 years ago
What about time I spent reporting this issue and finding the reason?

Original comment by xpeh.o...@googlemail.com on 24 Dec 2014 at 4:04

GoogleCodeExporter commented 8 years ago
I sometimes release windows binaries for releases. I probably will for the next 
release too. In the meantime you are welcome to check out the code from the 
repository and build it yourself.

Original comment by arvid.no...@gmail.com on 24 Dec 2014 at 6:51

GoogleCodeExporter commented 8 years ago
Why haven't you marked those binaries as deprecated? Why haven't you told I'm 
using outdated binaries?

Original comment by xpeh.o...@googlemail.com on 24 Dec 2014 at 7:31

GoogleCodeExporter commented 8 years ago
the files that are up as the default downloads on source forge are still the 
latest releases. they're only "deprecated" once they are replaced by a newer 
release.

if you consider any libtorrent based on an older version than head of the 
branch "outdated", you're likely to use outdated binaries more or less all the 
time. If you want to use the latest revision from svn, you'd need to build it 
yourself and keep pulling it.

Original comment by arvid.no...@gmail.com on 24 Dec 2014 at 8:18

GoogleCodeExporter commented 8 years ago
The newest binaries version was 0.16.10 while there is already sources release 
for 1.x. That's what I mean outdated. While you hidden Downloads, they are 
still accessible from google.

Original comment by xpeh.o...@googlemail.com on 25 Dec 2014 at 3:10

GoogleCodeExporter commented 8 years ago
what do you mean by 0.16.10 being the newest binaries? 1.0.3 is the latest 
stable and there are python binaries for windows. where do you see 0.16.10 as 
the newest binaries?

Original comment by arvid.no...@gmail.com on 25 Dec 2014 at 3:18

GoogleCodeExporter commented 8 years ago
Here https://code.google.com/p/libtorrent/downloads/list
As I said, it's still available from google.

Where are latest python binaries? 

Original comment by xpeh.o...@googlemail.com on 26 Dec 2014 at 11:40

GoogleCodeExporter commented 8 years ago
Do you mean this 
http://sourceforge.net/projects/libtorrent/files/py-libtorrent/ ? For some 
reason, google doesn't find them 
https://www.google.com/search?q=libtorrent+1.0.3+windows

Then please either delete files from 
https://code.google.com/p/libtorrent/downloads/list or mark them as deprecated 
(when you try to delete a file google code asks you if you want to mark it as 
deprecated instead). You should also take a look on this 
https://www.google.com/search?q=libtorrent+python+download - 
http://code.google.com/p/libtorrent/downloads/list is the first link for me.

Why haven't you told I'm using outdated verion? I've provided the download 
links.

Original comment by xpeh.o...@googlemail.com on 26 Dec 2014 at 11:58

GoogleCodeExporter commented 8 years ago
I've removed the downloads tab on the google project when I switched back to 
source forge. There's no way to navigate there. I suppose google keeps the 
links around for archival purposes and to not invalidate links that may exist 
to those files.

at libtorrent.org the python downloads point to source forge. Google search 
should be able to down-rank the hits of the removed downloads, but I suppose it 
doesn't.

Original comment by arvid.no...@gmail.com on 26 Dec 2014 at 1:02

GoogleCodeExporter commented 8 years ago
>I've removed the downloads tab on the google project when I switched back to 
source forge. There's no way to navigate there.
But Google (and probably other search engines) doesn't care :) So would you 
please mark all downloads as deprecated?

Is the link to google code downloads the first in google results because... 
it's google?

Sorry, but this issue is not fixed. Python 2.7.5 (default, May 15 2013, 
22:43:36) [MSC v.1500 32 bit (Intel)], Libtorrent 1.0.3.0 - same problem.

Original comment by xpeh.o...@googlemail.com on 26 Dec 2014 at 1:29

GoogleCodeExporter commented 8 years ago
Would you fix it please?

Original comment by xpeh.o...@googlemail.com on 2 Jan 2015 at 4:27

GoogleCodeExporter commented 8 years ago
[deleted comment]
GoogleCodeExporter commented 8 years ago
WOULD YOU OPEN THIS BUG??? IT'S NOT FIXED!!!!1

Original comment by xpeh.o...@googlemail.com on 21 Feb 2015 at 4:49

GoogleCodeExporter commented 8 years ago
I have the same problem. I am using Python version 2.66, 2.79 and libtorrent 
version 0.16.5, 0.16.8, 0.16.16, 1.03 on windows with the same result as above. 
But I can run it from Linux box using python 2.7.3 and 0.16.9. I think it is 
something with the windows release. Please fix it.

Original comment by umars...@gmail.com on 30 Mar 2015 at 10:34

GoogleCodeExporter commented 8 years ago
I think I've solved the problem. I am using python 2.7.9 and libtorrent 1.0.3 
on Windows 8.1 64 bit. The solution is written in 
http://stackoverflow.com/a/6930555 as noted above.

Using the source code at the top, what I did was just changing from

'save_path': './',

to 

'save_path': '.\\',

and it worked. I think the sample code at 
http://www.libtorrent.org/python_binding.html should be edited to use portable 
function such as os.getcwd() to avoid wasting time debugging.

Original comment by umars...@gmail.com on 31 Mar 2015 at 3:30

GoogleCodeExporter commented 8 years ago
Thanks. I will update the example.

Original comment by arvid.no...@gmail.com on 31 Mar 2015 at 10:16

GoogleCodeExporter commented 8 years ago
umars...@gmail.com

Why does it happen? Windows allows / as path separator.

Original comment by xpeh.o...@googlemail.com on 2 May 2015 at 11:51

GoogleCodeExporter commented 8 years ago
not if you use UNC paths. i.e. prefix it with "\\.\", which is also the only 
way to support long filenames/paths, and to support historically reserved words 
like "con", "prn" etc.

Original comment by arvid.no...@gmail.com on 3 May 2015 at 12:58

GoogleCodeExporter commented 8 years ago
>which is also the only way to support long filenames/paths
Wut?

How does cmd support long file names? It accepts / as separator too.

Why do you want to create files with reserved names? They won't be accessible 
from most programs.

Original comment by xpeh.o...@googlemail.com on 3 May 2015 at 2:40

GoogleCodeExporter commented 8 years ago
"long names" means 32000 characters. it's not libtorrent that decides the 
filenames, the .torrent file creators do. A .torrent file with a reserved name 
would otherwise hang the client (because it would try to access a device that 
doesn't exist, or doesn't behave like a file)

Original comment by arvid.no...@gmail.com on 3 May 2015 at 10:30

GoogleCodeExporter commented 8 years ago
And now will it just create a file that wouldn't be accessible from other 
programs?

Can you provide the code that doesn't accept / ?

Original comment by xpeh.o...@googlemail.com on 9 May 2015 at 12:29

GoogleCodeExporter commented 8 years ago
other (windows) programs have access to the same API libtorrent does. of course 
other programs can access files too.

what do you mean by "providing the code"?

When specifying paths in the UNC form on windows, *windows* does not accept 
forward slashes as directory separators. I don't have the windows source code, 
but that would be the code not accepting it.

You can read more about UNC paths here: 
https://msdn.microsoft.com/en-us/library/gg465305.aspx

Original comment by arvid.no...@gmail.com on 9 May 2015 at 1:21

GoogleCodeExporter commented 8 years ago
I mean the code in your lib that fails opening.

Original comment by xpeh.o...@googlemail.com on 11 May 2015 at 1:01

GoogleCodeExporter commented 8 years ago
most (all I believe) of disk I/O abstracted by src/file.cpp, providing a 
platform independent interface.

Original comment by arvid.no...@gmail.com on 11 May 2015 at 6:09