danielfrg / s3contents

Jupyter Notebooks in S3 - Jupyter Contents Manager implementation
Apache License 2.0
248 stars 88 forks source link

Unable to move a filesystem dir into s3 #112

Open erowan opened 3 years ago

erowan commented 3 years ago

Hello

I am using hybridcontents to view a real filesystem combined with the virtual s3 filesystem with s3contents. I can drag and drop single files but am unable to move a directory with contents. Is this a known limitation?

I attach the log with corresponding error.

[D 2021-01-05 15:06:47.444 u038178 auth:857] Allowing Hub admin u038178                                                                                
[D 2021-01-05 15:06:47.445 u038178 auth:744] Setting oauth cookie for 100.127.74.131: jupyterhub-user-u038178, {'path': '/user/u038178/', 'httponly': T
rue}                                                                                                                                                   
[D 2021-01-05 15:06:47.446 u038178 genericmanager:250] S3contents.GenericManager.save {'name': 'testcp', 'path': 'testcp', 'last_modified': datetime.da
tetime(2021, 1, 5, 15, 0, 33, 200399, tzinfo=<notebook._tz.tzUTC object at 0x7fbd42ce4390>), 'created': datetime.datetime(2021, 1, 5, 15, 6, 41, 329752
, tzinfo=<notebook._tz.tzUTC object at 0x7fbd42ce4390>), 'content': [{'name': 'untitled1.txt', 'path': 'testcp/untitled1.txt', 'last_modified': datetim
e.datetime(2021, 1, 5, 15, 0, 33, 200399, tzinfo=<notebook._tz.tzUTC object at 0x7fbd42ce4390>), 'created': datetime.datetime(2021, 1, 5, 15, 0, 33, 20
0399, tzinfo=<notebook._tz.tzUTC object at 0x7fbd42ce4390>), 'content': None, 'format': None, 'mimetype': 'text/plain', 'size': 0, 'writable': True, 't
ype': 'file'}, {'name': 'untitled.txt', 'path': 'testcp/untitled.txt', 'last_modified': datetime.datetime(2021, 1, 5, 15, 0, 28, 457007, tzinfo=<notebo
ok._tz.tzUTC object at 0x7fbd42ce4390>), 'created': datetime.datetime(2021, 1, 5, 15, 0, 28, 457007, tzinfo=<notebook._tz.tzUTC object at 0x7fbd42ce439
0>), 'content': None, 'format': None, 'mimetype': 'text/plain', 'size': 0, 'writable': True, 'type': 'file'}], 'format': 'json', 'mimetype': None, 'siz
e': None, 'writable': True, 'type': 'directory'}: 'testcp'                                                                                             
[D 2021-01-05 15:06:47.446 u038178 s3_fs:247] S3FS.unprefix: self.prefix_: g0h2iz9h-mercurytest path: ['testcp', '.s3keep']                            
[D 2021-01-05 15:06:47.446 u038178 s3_fs:174] S3contents.S3FS: Making dir: `g0h2iz9h-mercurytest/testcp/.s3keep`                                       
[D 2021-01-05 15:06:47.495 u038178 genericmanager:112] S3contents.GenericManager.get] path('testcp') type(directory) format(None)                      
[D 2021-01-05 15:06:47.495 u038178 genericmanager:134] S3contents.GenericManager._get_directory: path('testcp') content(False) format(None)            
[D 2021-01-05 15:06:47.495 u038178 genericmanager:160] S3contents.GenericManager._directory_model_from_path: path('testcp') type(False)                
[D 2021-01-05 15:06:47.495 u038178 s3_fs:247] S3FS.unprefix: self.prefix_: g0h2iz9h-mercurytest path: ['testcp']                                       
[D 2021-01-05 15:06:47.503 u038178 s3_fs:139] S3contents.S3FS: `g0h2iz9h-mercurytest/testcp` is a directory: True                                      
[D 2021-01-05 15:06:47.503 u038178 s3_fs:247] S3FS.unprefix: self.prefix_: g0h2iz9h-mercurytest path: ['testcp']                                       
[D 2021-01-05 15:06:47.503 u038178 s3_fs:247] S3FS.unprefix: self.prefix_: g0h2iz9h-mercurytest path: ['testcp', '.s3keep']                            
[W 2021-01-05 15:06:47.510 u038178 web:1667] 400 PATCH /user/u038178/api/contents/testcp?1609859207139 (100.127.74.131): Directory /home/jovyan/testcp 
not empty                                                                                                                                              
[W 2021-01-05 15:06:47.510 u038178 handlers:608] Directory /home/jovyan/testcp not empty    
danielfrg commented 3 years ago

Probably a limitation on the ContentManager or S3FS. So its possible this will not work.