Closed AaronDavidSchneider closed 3 years ago
Urgh. I was afraid of something like this. I was hoping that a version number > 3.0 on MacFuse indicated that it supported FUSE3, but apparently not. From those issues, I don't hold much hope of MacFuse ever making it to FUSE3.
I don't think there's anything I'm doing that requires FUSE3 specifically; that requirement is coming from the fact that I chose to use pyfuse3, which seemed to be the most active and best documented library. There were a bunch of other bindings I found out there, but the only other one I seriously considered was python-fuse. This is for FUSE2, but I don't know whether it works with MacFuse. If someone would like to check and let me know, we could see if converting to the python-fuse library makes sense. (A simple test would be to pip install fuse-python
and then see if this example works.)
All that said, this particular error was because the pyfuse3 module needs to be compiled during installation, and it couldn't find the fuse3 headers. The fuse-python module is the same way, so I don't know if this would also hit the same problem.
Yes I think libfuse2 should work with osx. I tested the example script that you provided (after I successfully installed fuse-python). I get an error,
fuse: no mount point
Traceback (most recent call last):
File "hello.py", line 91, in <module>
main()
File "hello.py", line 88, in main
server.main()
File "/Users/schneider/anaconda3/lib/python3.8/site-packages/fuse.py", line 754, in main
main(**d)
fuse.FuseError: filesystem initialization failed
which seems to be related to the example script, rather than fuse itself. However, since
import fuse
from fuse import Fuse
works. I have no doubt that using libfuse2 would fix the problem for OSX.
However, I suppose that libfuse3 is more stable.... Its a pity that macfuse did not update libfuse....
Thanks for checking. It appears that pyfuse3 forked off from python-llfuse some time ago. That project is now marked as undeveloped, but the API is still pretty similar. It has some hints on building solutions compatible with both libraries. I'll take a look and see if that's feasible here.
Good news -- it seems to work. See #6 for a branch that implements support for llfuse. If you could try it out and let me know if it works with MacFuse, it'd be much appreciated. You'll need llfuse (pip install llfuse
). If you're working from git, you can just pull the branch from #6. If not, you should be able to pip install
the attached sdist file.
That sounds awesome! I will test it today!
It installed successfully! These steps were necessary:
brew install macfuse
One needs to allow a kernel extension and reboot.
brew install pkg-config
I messed up the onetime code and now get the error:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/schneider/.cache/rmfuse'
Seems to be unrelated to rmfuse.
That is a bug in RMfuse--it stores file metadata in a cache database to avoid hitting the API too often. But I didn't make sure the ~/.cache
directory existed before trying to create the db file. I'll get a fix in for that soon, but in the meantime, you ought to be able to fix this with a mkdir ~/.cache
on the command line.
thanks for getting back!
I started rmfuse
:
(rmfuse) schneider@Aarons-MacBook-Pro ~ % rmfuse ~/remarkable -v
DEBUG:llfuse:Initializing llfuse
DEBUG:llfuse:Calling fuse_mount
DEBUG:llfuse:Calling fuse_lowlevel_new
DEBUG:llfuse:Calling fuse_session_add_chan
DEBUG:llfuse:Calling fuse_session_loop
It doesn't finish the session. A new window yields:
(base) schneider@Aarons-MBP ~ % cd remarkable
(base) schneider@Aarons-MBP . % ls
ls: .: Function not implemented
(base) schneider@Aarons-MBP . % open .
2021-02-14 12:52:14.104 open[1833:36366] CFURLCopyResourcePropertyForKey failed because it was passed an URL which has no scheme
No application knows how to open . (Error Domain=NSOSStatusErrorDomain Code=-10814 "kLSApplicationNotFoundErr: E.g. no application claims the file" UserInfo={_LSLine=1483, _LSFunction=runEvaluator}).
(base) schneider@Aarons-MBP . % cp ~/Desktop/Chapter_3.pdf ~/remarkable
cp: /Users/schneider/remarkable: Function not implemented
Don't get confused by (base)
or (rmfuse)
. Thats just due to the conda environment that I used to setup rmfuse
.
Second question first: the tokens are stored in ~/.config/rmcl/config.json
. Delete that file, and you'll be back to pristine, as far as the API is concerned. Then you can get a new one-time code and get access again.
As for the first problem: it appears that your system is trying to call a filesystem method that isn't implemented in RMfuse yet. (There are quite a few, and I implemented them as I found they were needed in testing.) It's not clear to me which one it is, but if you run rmfuse
with the -vv
output, you'll get a whole lot of debugging information, including each function being called. If you can capture that output and post it here, it'll help narrow down what the problem is.
Thanks!
Thanks! Here we go:
I see a lot of failed calls to statfs, which hadn't been implemented. I've added a simple implementation in #6. Can you give it a try. Updated sdist file attached, if that helps. rmfuse-0.1.2a0.tar.gz
Awesome! it works for copying stuff from the fuse drive to the local machine. But right now its giving me an error, when I try to copy a pdf to the fuse drive. Here is a new log:
The error seems to be that RMfuse cannot determine that the file you're trying to upload is a PDF or EPUB file. Which type is it? (Note that if it's not one of these, the upload should fail.)
If you're able to send me the file, I can take a look. If not, just the beginning of the file would be enough for me to see if RMfuse should be able to recognize it. You can run head -c 100 file.pdf > file.fragment
, for example, to get the first 100 bytes of file.pdf for me.
Hi, I tested it with the same file that I downloaded from the remarkable. It was definitely a pdf file.
Here is again the python error trace when I try to cp
:
(rmfuse) schneider@Aarons-MacBook-Pro ~ % cp ~/Desktop/debug.pdf remarkable
cp: remarkable/debug.pdf: fchmod failed: Input/output error
Traceback (most recent call last):
File "/Users/schneider/anaconda3/envs/rmfuse/bin/rmfuse", line 8, in <module>
sys.exit(main())
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuse.py", line 470, in main
fuse.main(workers=1)
File "src/fuse_api.pxi", line 327, in llfuse.main
File "src/handlers.pxi", line 136, in llfuse.fuse_setattr
File "src/handlers.pxi", line 137, in llfuse.fuse_setattr
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuselib.py", line 25, in decorated
return trio.run(runner)
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/trio/_core/_run.py", line 1932, in run
raise runner.main_task_outcome.error
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuselib.py", line 24, in runner
return await afunc(*args, **kw)
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuse.py", line 232, in setattr
log.debug(f'setattr called on {await self.get_by_id(self.get_id(inode))!r}')
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuse.py", line 139, in get_by_id
return await Item.get_by_id(id_)
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmcl/items.py", line 46, in get_by_id
return await (await api.get_client()).get_by_id(id_)
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmcl/api.py", line 247, in get_by_id
return self.by_id[id_]
KeyError: 'c369d328-db7f-4673-8588-ac0305b8fbad'
cp: remarkable/debug.pdf: Device not configured
[1] + exit 1 rmfuse remarkable
cp: /Users/schneider/Desktop/debug.pdf: could not copy extended attributes to remarkable/debug.pdf: Device not configured
Here is another error which I get upon renaming a file:
(rmfuse) schneider@Aarons-MacBook-Pro remarkable % mv How\ to\ Debug\ Fortran\ code\ with\ GDB\ -\ Fortran\ Debugging\ -\ Undo.io.pdf debug.pdf
mv: rename How to Debug Fortran code with GDB - Fortran Debugging - Undo.io.pdf to debug.pdf: Input/output error
(rmfuse) schneider@Aarons-MacBook-Pro remarkable % Traceback (most recent call last):
File "/Users/schneider/anaconda3/envs/rmfuse/bin/rmfuse", line 8, in <module>
sys.exit(main())
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuse.py", line 470, in main
fuse.main(workers=1)
File "src/fuse_api.pxi", line 327, in llfuse.main
File "src/handlers.pxi", line 263, in llfuse.fuse_rename
File "src/handlers.pxi", line 264, in llfuse.fuse_rename
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuselib.py", line 25, in decorated
return trio.run(runner)
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/trio/_core/_run.py", line 1932, in run
raise runner.main_task_outcome.error
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuselib.py", line 24, in runner
return await afunc(*args, **kw)
TypeError: rename() missing 1 required positional argument: 'ctx'
Here is the debug.pdf that I copied from the rmcloud to my desktop. And which I tried to upload back to the rm. On the rm this file is called How\ to\ Debug\ Fortran\ code\ with\ GDB\ -\ Fortran\ Debugging\ -\ Undo.io.pdf
debug.pdf
Thanks!
Thanks. I think I missed the tracebacks into the logs above. The good new is, both of these are simple bugs. I've pushed fixes; give it a try. (I swear we're getting closer, even if it doesn't seem that way!)
Thanks again for debugging! We are getting there!
Renaming and deleting files seems to work now. Moving files (either by shell command or by finder) does not yet work.
It often boils down to these errors: RELEASEDIR
and LOOKUP
.
unique: 17, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 434
unique: 17, error: -78 (Function not implemented), outsize: 16
unique: 18, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 434
unique: 18, success, outsize: 136
unique: 19, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 434
unique: 19, success, outsize: 96
unique: 2, opcode: LOOKUP (1), nodeid: 1, insize: 60, pid: 65
unique: 2, error: -2 (No such file or directory), outsize: 16
Both give always errors.
Edit: Is there a reason why rmfuse runs in the foreground by default? Its not a problem to do something like rmfuse ~/remarkable &
but wouldn't it be more elegant to have it run in the background by default?
I'm not convinced that those errors are a problem. The LOOKUP is probably result of something on your system looking for a file that doesn't exist, perhaps some metadata directory. The RELEASEDIR isn't implemented, since RMfuse doesn't have anything to do to release a directory. But that shouldn't cause problems. (We can try adding a stub if we can't figure out anything else, but I suspect it wouldn't really help.)
What's the failure mode of moving the file? Does it crash? If so, please provide the traceback. Does it hang? How long did you wait? Does nothing happen? Does this happen when moving files within the RMfuse mount point, moving a file from your system to RMfuse, or moving a file from RMfuse to your system?
sorry for the poor Bugreport. Let me try to improve on your questions:
It oddly depends on the file (and its size). I tested four files:
CMD
+ P
), size=200KB What's the failure mode of moving the file? In the file explorer (hereafter finder) I always get this error (regardless of the file):
The operation could not be completed because at least one required object was not found (error -43).
Copying using the terminal succeeds for the smaller documents (ms2.pdf
and rmfuse_test.pdf
).
In terminal that is moving the file:
cp: Desktop/paper_2/ms2.pdf: could not copy extended attributes to /Users/schneider/remarkable/ms2.pdf: Attribute not found
In terminal that is in control of rmfuse
:
Error: Not a PDF or EPUB file
However the two larger documents do not succeed:
(base) schneider@Aarons-MacBook-Pro Desktop % cp Molliere_thesis.pdf ~/remarkable
cp: Molliere_thesis.pdf: could not copy extended attributes to /Users/schneider/remarkable/Molliere_thesis.pdf: Operation not supported
in rmfuse window:
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file
ERROR:root:Attempt to get non-existant Item 699b20b3-6826-4625-b22a-600354ee4e38
Does it crash? If so, please provide the traceback. Does it hang?
rmfuse does not crash nor hang.
How long did you wait?
a couple of minutes. I also checked with rmapi if the file was copied but just not listed.
Does this happen when moving files within the RMfuse mount point, moving a file from your system to RMfuse, or moving a file from RMfuse to your system?
Moving a file inside rmfuse and moving a file from rmfuse to system gives no error. Only moving files from local to rmfuse.
Thanks, that's very informative. I just don't quite know what it all means. :)
The errors about the extended attributes are expected -- RMfuse doesn't implement them right now. (It might be possible to stuff them into the document metadata at some point.) But it is weird that the error on setting them is different in the two cases. If they get annoying, we could add an option to RMfuse that would pretend to set the attributes, which should get rid of that message.
I'm not sure what's going wrong with the "Not a PDF or EPUB file" error. My best guess is that there's some metadata file that the Mac is trying to create / copy along with the PDF file. The fact that it's happening in both successes and failures suggests that it's not a fatal problem.
The worrisome bit is the "Attempt to get non-existent Item". That means something is going wrong in the upload process. I wonder if we have a race condition -- something is trying to do something to a file while it's in the process of being uploaded. This would slip in a gap of awareness of RMfuse -- it can't find the document in the set of editable new-created documents, since we closed it out to upload. But it also doesn't see it in the list of documents from the Cloud, because the upload hasn't finished. (This would explain why it's only happening to larger files.)
I've built a new version with some more debugging output and an attempt to handle the above case. Please give it a try (with -v
) and let me know what happens. One trick I've used for keeping track of what's happening: If you press 'Enter' in the rmfuse terminal between events you're triggering in another terminal, you can better associate the debugging lines with specific actions.
Thanks for the explainations!
Copying Molliere_thesis or ms2 to rmfuse using finder (still same error code -43 in finder):
DEBUG:rmfuse.fuse:Created Molliere_thesis for b'Molliere_thesis.pdf', with inode 16 and ID 9dc8cf3f-17ec-40e0-80e1-f4b61ea41289
DEBUG:rmfuse.fuse:Releasing inode 16
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was Molliere_thesis)
ERROR:root:Attempt to get non-existent Item 9dc8cf3f-17ec-40e0-80e1-f4b61ea41289
The same from command line (only the last couple lines...):
(...)
DEBUG:rmfuse.fuse:Writing to 18 at 30257152, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30261248, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30265344, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30269440, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30273536, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30277632, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30281728, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30285824, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30289920, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30294016, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30298112, length 755
DEBUG:rmfuse.fuse:setattr called on <Document "Molliere_thesis">
DEBUG:rmfuse.fuse: Attempting to set mode to 33188
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:setattr called on <Document "Molliere_thesis">
DEBUG:rmfuse.fuse: Attempting to set mode to 33188
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:Created ._Molliere_thesis for b'._Molliere_thesis.pdf', with inode 19 and ID 870c59c6-47aa-42a1-8599-291f5e2815e9
DEBUG:rmfuse.fuse:Writing to 19 at 0, length 4096
DEBUG:rmfuse.fuse:Releasing inode 19
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was ._Molliere_thesis)
DEBUG:rmfuse.fuse:Releasing inode 18
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was Molliere_thesis)
ERROR:root:Attempt to get non-existent Item 4a4313ac-4c86-493d-9f5b-2378666aa553
In contrast here we have ms2.pdf which works:
DEBUG:rmfuse.fuse:Created ms2 for b'ms2.pdf', with inode 20 and ID 6e376b45-4c4b-4e4b-a5b1-34e9679da343
DEBUG:rmfuse.fuse:setattr called on <Document "ms2">
DEBUG:rmfuse.fuse: Attempting to set mode to 33152
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:Writing to 20 at 0, length 4096
DEBUG:rmfuse.fuse:Writing to 20 at 4096, length 4096
DEBUG:rmfuse.fuse:Writing to 20 at 8192, length 4096
(...)
DEBUG:rmfuse.fuse:Writing to 20 at 950272, length 4096
DEBUG:rmfuse.fuse:Writing to 20 at 954368, length 4096
DEBUG:rmfuse.fuse:Writing to 20 at 958464, length 2067
DEBUG:rmfuse.fuse:setattr called on <Document "ms2">
DEBUG:rmfuse.fuse: Attempting to set mode to 33188
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:setattr called on <Document "ms2">
DEBUG:rmfuse.fuse: Attempting to set mode to 33188
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:Created ._ms2 for b'._ms2.pdf', with inode 21 and ID 539b5949-8baa-4336-9c60-5febbb25dfe3
DEBUG:rmfuse.fuse:Writing to 21 at 0, length 4096
DEBUG:rmfuse.fuse:Releasing inode 21
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was ._ms2)
DEBUG:rmfuse.fuse:Releasing inode 20
DEBUG:rmcl.items:Getting details for <Document "ms2">
DEBUG:rmcl.items:Details for <Document "ms2">: type pdf, size 960531
ERROR:root:Attempt to get non-existent Item 539b5949-8baa-4336-9c60-5febbb25dfe3
From what I can see:
The difference in cp
from terminal between Molliere and ms2 boils down to these two lines missing for Molliere_thesis:
DEBUG:rmcl.items:Getting details for <Document "ms2">
DEBUG:rmcl.items:Details for <Document "ms2">: type pdf, size 960531
The difference between a finder copy and a terminal copy seems to be everything in between the first line and the last few lines.
Thanks. I feel like we're inching towards understanding. The thing that has me worried is this:
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was Molliere_thesis) This means that RMfuse couldn't figure out that the file is a PDF document. Therefore, it couldn't upload the file, and that's why nothing's working.
It looks like there may be two different failures, unfortunately. With finder, it appears that no data was written to the file. I'm not sure why that is. (Maybe finder planned on re-opening the file to write the contents later?) But it makes sense that it failed. But from the command line, we can see a lot of data did get written, so I'm not sure what went wrong there.
RMfuse assumes that a PDF file starts with '%PDF'. Can you check whether this is the case for your problematic files? (The 'head' command is useful for reading the first n bytes of a file.) If it's not, can you include the first couple of KB of those files so I can take a look at what's going on there?
On Sun, Feb 21, 2021 at 11:23 PM Aaron David Schneider < notifications@github.com> wrote:
Thanks for the explainations!
Copying Molliere_thesis or ms2 to rmfuse using finder (still same error code -43 in finder):
DEBUG:rmfuse.fuse:Created Molliere_thesis for b'Molliere_thesis.pdf', with inode 16 and ID 9dc8cf3f-17ec-40e0-80e1-f4b61ea41289 DEBUG:rmfuse.fuse:Releasing inode 16 ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was Molliere_thesis) ERROR:root:Attempt to get non-existent Item 9dc8cf3f-17ec-40e0-80e1-f4b61ea41289
The same from command line (only the last couple lines...):
(...) DEBUG:rmfuse.fuse:Writing to 18 at 30257152, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30261248, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30265344, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30269440, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30273536, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30277632, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30281728, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30285824, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30289920, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30294016, length 4096 DEBUG:rmfuse.fuse:Writing to 18 at 30298112, length 755 DEBUG:rmfuse.fuse:setattr called on <Document "Molliere_thesis"> DEBUG:rmfuse.fuse: Attempting to set mode to 33188 DEBUG:rmfuse.fuse: No changes made DEBUG:rmfuse.fuse:setattr called on <Document "Molliere_thesis"> DEBUG:rmfuse.fuse: Attempting to set mode to 33188 DEBUG:rmfuse.fuse: No changes made DEBUG:rmfuse.fuse:Created ._Molliere_thesis for b'._Molliere_thesis.pdf', with inode 19 and ID 870c59c6-47aa-42a1-8599-291f5e2815e9 DEBUG:rmfuse.fuse:Writing to 19 at 0, length 4096 DEBUG:rmfuse.fuse:Releasing inode 19 ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was ._Molliere_thesis) DEBUG:rmfuse.fuse:Releasing inode 18 ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was Molliere_thesis) ERROR:root:Attempt to get non-existent Item 4a4313ac-4c86-493d-9f5b-2378666aa553
In contrast here we have ms2.pdf which works:
DEBUG:rmfuse.fuse:Created ms2 for b'ms2.pdf', with inode 20 and ID 6e376b45-4c4b-4e4b-a5b1-34e9679da343 DEBUG:rmfuse.fuse:setattr called on <Document "ms2"> DEBUG:rmfuse.fuse: Attempting to set mode to 33152 DEBUG:rmfuse.fuse: No changes made DEBUG:rmfuse.fuse:Writing to 20 at 0, length 4096 DEBUG:rmfuse.fuse:Writing to 20 at 4096, length 4096 DEBUG:rmfuse.fuse:Writing to 20 at 8192, length 4096 (...) DEBUG:rmfuse.fuse:Writing to 20 at 950272, length 4096 DEBUG:rmfuse.fuse:Writing to 20 at 954368, length 4096 DEBUG:rmfuse.fuse:Writing to 20 at 958464, length 2067 DEBUG:rmfuse.fuse:setattr called on <Document "ms2"> DEBUG:rmfuse.fuse: Attempting to set mode to 33188 DEBUG:rmfuse.fuse: No changes made DEBUG:rmfuse.fuse:setattr called on <Document "ms2"> DEBUG:rmfuse.fuse: Attempting to set mode to 33188 DEBUG:rmfuse.fuse: No changes made DEBUG:rmfuse.fuse:Created ._ms2 for b'._ms2.pdf', with inode 21 and ID 539b5949-8baa-4336-9c60-5febbb25dfe3 DEBUG:rmfuse.fuse:Writing to 21 at 0, length 4096 DEBUG:rmfuse.fuse:Releasing inode 21 ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was ._ms2) DEBUG:rmfuse.fuse:Releasing inode 20 DEBUG:rmcl.items:Getting details for <Document "ms2"> DEBUG:rmcl.items:Details for <Document "ms2">: type pdf, size 960531 ERROR:root:Attempt to get non-existent Item 539b5949-8baa-4336-9c60-5febbb25dfe3
From what I can see: The difference in cp from terminal between Molliere and ms2 boils down to these two lines missing for Molliere_thesis:
DEBUG:rmcl.items:Getting details for <Document "ms2"> DEBUG:rmcl.items:Details for <Document "ms2">: type pdf, size 960531
The difference between a finder copy and a terminal copy seems to be everything in between the first line and the last few lines.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rschroll/rmfuse/issues/2#issuecomment-783151912, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACZ2HZQYXJG545Q2WXCKMLTAIA7DANCNFSM4XISMUSA .
Thanks for your quick reply!
Here is the head output of Molliere thesis:
%PDF-1.3
%����
1 0 obj
<</CreationDate 8963 0 R /Creator 8962 0 R /ModDate 8963 0 R /Producer 8961 0 R >>
endobj
2 0 obj
<</Count 13/First 7488 0 R /Last 7489 0 R >>
endobj
3 0 obj
<</Contents 5 0 R /MediaBox[ 0 0 595.276 841.89]/Parent 4 0 R /Resources 7 0 R /Rotate 0/Type/Page>>
I uploaded a test file (an old presentation I gave during my undergrad): rmfuse_test.pdf
We could use this file as a benchmark for this issue since this one fails too and I can share it.
The bad (or maybe good?) news is that that NASA PDF file works just fine on my test system. This suggests to me that the Mac is doing something I didn't anticipate when writing the file.
Can you check through the 'Writing to X at Y, length Z' messages? Does Y start at zero and always increase? Or does it jump around. (On my system, it increases steadily, but if it doesn't I can imagine that it might break things. Though I don't see how it can break the beginning of the file.)
The only other thought I have is to look at the actual values RMfuse finds in the file. If you're willing to get your hands dirty, try adding a line to rmfuse/fuse.py. Right after the line
log.error(f'Error: Not a PDF or EPUB file (file was {document.name})')
Try adding
log.error(f'File header: {data[:1024]}')
This will print out the first KB of the file as RMfuse got it, and we can see if it's different from what it should be. If you're not comfortable mucking about like this, I can create an instrumented version with this and other debug output later.
I just tested your second idea with the Nasa file.
Upon
(base) schneider@Aarons-MacBook-Pro ~ % cp Downloads/A17_FlightPlan.pdf ~/remarkable
cp: Downloads/A17_FlightPlan.pdf: could not copy extended attributes to /Users/schneider/remarkable/A17_FlightPlan.pdf: Operation not supported
I get the following output in the rmfuse window:
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was ._A17_FlightPlan)
ERROR:rmfuse.fuse:File header: b'\x00\x05\x16\x07\x00\x02\x00\x00Mac OS X \x00\x02\x00\x00\x00\t\x00\x00\x002\x00\x00\x0e\xb0\x00\x00\x00\x02\x00\x00\x0e\xe2\x00\x00\x01\x1e\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ATTR\x00\x00\x00\x12\x00\x00\x0e\xe2\x00\x00\x00x\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was A17_FlightPlan)
ERROR:rmfuse.fuse:File header: b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
ERROR:root:Attempt to get non-existent Item 0e77f5b9-e23f-423b-9fa9-a38a522c8b59
It looks like it is trying to copy two files. The second one has only \x00
in it and the first one has at least a different beginning.
I might be mistaken, but is that really the content of the pdf???
But I need to admit that I do not know much about binary files... other than that they are nasty to deal with ...
Yeah, it looks like the file it is trying to upload is full of null bytes. That ain't right. (As we saw above, the PDF file should start with '%PDF'.)
Unfortunately, I have no idea why that would be the case. Perhaps the OS decided it would fill the file with zeros first, and then write the actual content? But I don't know why that would be the case, or why that would only happen with large files.
Another possibility would be a bug with llfuse or Macfuse. But again, I don't know why that would only affect large files.
I'll have to puzzle over this for a while.
In happier news, those ._
files are apparently for filesystem metadata. It appears you can turn that off with a special fuse option. If you add fuse_options.add('noappledouble')
around here, it might stop complaining about those files. But I don't think it has any effect on the real problem here.
No hurry 👌
In happier news, those ._ files are apparently for filesystem metadata. It appears you can turn that off with a special fuse option. If you add fuse_options.add('noappledouble') around here, it might stop complaining about those files. But I don't think it has any effect on the real problem here.
I can confirm.
No brilliant thoughts so far. Just to confirm: Other than uploading large files, things seem to be working pretty well? I'm thinking that maybe I should merge #6 into master and cut a new release, with the hope that additional users will help us figure out what's going wrong.
How does that sit with you?
I've added MacOS install directions, based on your descriptions here. Can you check that I got the correct? https://github.com/rschroll/rmfuse/blob/release-0.2.0/README.md#macos
Note that if you want to try this out yourself, you'll need to use the --pre
option to install a prerelease version of rmfuse with pip.
Thanks!
yes that sounds good!
The new version doesn't start on osx.
(rmfuse) schneider@Aarons-MacBook-Pro ~ % rmfuse ~/remarkable -v
DEBUG:llfuse:Initializing llfuse
DEBUG:llfuse:Calling fuse_mount
DEBUG:llfuse:Calling fuse_lowlevel_new
fuse: unknown option `nodoubleapple'
Traceback (most recent call last):
File "/Users/schneider/anaconda3/envs/rmfuse/bin/rmfuse", line 8, in <module>
sys.exit(main())
File "/Users/schneider/anaconda3/envs/rmfuse/lib/python3.9/site-packages/rmfuse/fuse.py", line 563, in main
fuse.init(fs, options.mountpoint, fuse_options)
File "src/fuse_api.pxi", line 253, in llfuse.init
RuntimeError: fuse_lowlevel_new() failed
The reason seems to be the typo: noappledouble
instead of nodoubleapple
.
Also: I can't install it the way you suggest it:
(rmfuse) schneider@Aarons-MacBook-Pro ~ % pip install rmfuse[llfuse]
zsh: no matches found: rmfuse[llfuse]
I suppose pip install llfuse rmfuse
should work instead?
On Sun, Feb 28, 2021 at 10:55 PM Aaron David Schneider < notifications@github.com> wrote:
The new version doesn't start on osx.
The reason seems to be the typo: noappledouble instead of nodoubleapple.
D'oh!
I can't install it the way you suggest it:
(rmfuse) schneider@Aarons-MacBook-Pro ~ % pip install rmfuse[llfuse] zsh: no matches found: rmfuse[llfuse]
You'll need to add the
--pre
option, to look for pre-release versions in pip.
I'll close this issue for now, and we can keep working on the large file upload problem in #18.
Hi,
I really love the idea of
rmfuse
and I would love to use it on my Mac.It seems that that
pyfuse3
is not supported on OSX: https://github.com/osxfuse/osxfuse/issues/390 https://github.com/libfuse/pyfuse3/issues/29This is the error I get when doing
pip install rmfuse