don't return ls - until directory structure is known

[saqebakhter]

having an empty ls returned can be misleading..

[EricTheMagician]

I don't disagree with that. I currently cannot tell apart the difference between an empty directory and one that hasn't loaded.

The solution I have is to update all folders periodically and save them and reload it when the drive is mounted again. The problem would still remain on first load, but should quickly disappear after some use.

Expect to see that coming soon.. It currently should be able to update folders every 15 minutes though the time it takes to update is highly variable from one account to another. On Aug 5, 2014 4:03 PM, "saqebakhter" notifications@github.com wrote:

having an empty ls returned can be misleading..

— Reply to this email directly or view it on GitHub https://github.com/thejinx0r/BitcasaF4JS/issues/3#issuecomment-51251524.

[saqebakhter]

Does it currently update periodically right now? On Aug 5, 2014 9:13 PM, "thejinx0r" notifications@github.com wrote:

I don't disagree with that. I currently cannot tell apart the difference between an empty directory and one that hasn't loaded.

The solution I have is to update all folders periodically and save them and reload it when the drive is mounted again. The problem would still remain on first load, but should quickly disappear after some use.

Expect to see that coming soon.. It currently should be able to update folders every 15 minutes though the time it takes to update is highly variable from one account to another. On Aug 5, 2014 4:03 PM, "saqebakhter" notifications@github.com wrote:

having an empty ls returned can be misleading..

— Reply to this email directly or view it on GitHub https://github.com/thejinx0r/BitcasaF4JS/issues/3#issuecomment-51251524.

— Reply to this email directly or view it on GitHub https://github.com/thejinx0r/BitcasaF4JS/issues/3#issuecomment-51282679.

[EricTheMagician]

it should be, i've pushed the commits

[saqebakhter]

With the latest code I always seem to get this :

{"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T04:12:24.034Z"}

Isn't 1 minute a little fast to update the folders?

From the looks of it , when getAllFolders() runs, my /mnt/BitcasaJS/Bitcasa Infinite Drive/Media - loses its structure, and does not get it until i 'cd' to that dir and do an 'ls'

example:

'ls' before getAllFolders()

sakhter@sakhter-ubuntu:/mnt/Bitcasa/Media$ ls Movies Movies2 TVShows TVShows_Finished TVShows_Other

{"level":"debug","message":"running get all folders","timestamp":"2014-08-06T05:46:26.682Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T05:46:42.858Z"}

getAllFolders runs....

do an 'ls'

sakhter@sakhter-ubuntu:/mnt/Bitcasa/Media$ ls sakhter@sakhter-ubuntu:/mnt/Bitcasa/Media$

triggers a refresh:

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T05:46:54.814Z"}

*/Media/ populates after 2-4 minutes - but i think it should update the structure in the background...and not leave it 'empty' until an 'ls' is run.

[EricTheMagician]

getAllFolders is run every 15 minutes. So, after 3 or 4 round, it should be fine. To speed it up a little, there's a different mechanism that is run when it detects an empty folder.

Initially, getAllFolders will run once on just the root folder because nothing has loaded yet, and then in 15 minutes, it will update all the folders recursively. With the way I stored my files, it takes about 6 minutes.

Like I said before: The solution I have is to update all folders periodically and save them and reload it when the drive is mounted again. The problem would still remain on the very first use, but should quickly disappear after some use.

[saqebakhter]

Hey,

Thanks for the info - yeah I see it trying to do that now - but It's also getting this error:

{"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token < - folders length - 38","timestamp":"2014-08-06T14:02:34.504Z"}

{"level":"debug","message":"it took 2 minutes to update folders","timestamp":"2014-08-06T14:03:16.786Z"}

Not sure if that would cause issues.

[EricTheMagician]

It might cause some issues in the future, but for now, it just fails to update a folder and it will retry. Since it says it completed in 2 minutes, it means that it retried and succeeded.

The future problem is that it might drop an entire folder and it's children. Not sure how to get around that other than retrying 5 times and then retrying again in 15 minutes.

[saqebakhter]

I think in my case it is dropping a folder, and its children - everytime it runs my /BitcasaFS/Bitcasa Infinite Drive/Media is empty - and doesn't populate until I do an LS.

[EricTheMagician]

That's odd, can you post more of your log? And is it just your media folder that is empty or is it all of the folders inside your infinite drive?

Eric

On Wed, Aug 6, 2014 at 11:44 AM, saqebakhter notifications@github.com wrote:

I think in my case it is dropping a folder, and its children - everytime it runs my /BitcasaFS/Bitcasa Infinite Drive/Media is empty - and doesn't populate until I do an LS.

— Reply to this email directly or view it on GitHub https://github.com/thejinx0r/BitcasaF4JS/issues/3#issuecomment-51353355.

[saqebakhter]

I've added some more logging commands and i'm running it again right now - i'll let it go through its paces a few times and post the log.

[saqebakhter]

I changed the refresh time to 5 minutes to make this go faster - i also tried to print the json - data variable thats what the - "json - undefined", not sure why it didn't print.

Went to /mnt/BitcasaFS - did an LS then waited:

{"level":"debug","message":"getting folder info from bitcasa for / -- depth 1","timestamp":"2014-08-06T16:07:11.302Z"} {"level":"info","message":"mount point: /mnt/BitcasaJS","timestamp":"2014-08-06T16:07:11.321Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:07:22.055Z"}

{"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token u - folders length - 38","timestamp":"2014-08-06T16:12:31.148Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.149Z"} {"level":"error","message":"there was a problem processing i=1(SuperPuTTY) - SyntaxError: Unexpected token u - folders length - 39","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"there was a problem processing i=2(MyJabberFiles) - SyntaxError: Unexpected token u - folders length - 40","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"there was a problem processing i=3(cache) - SyntaxError: Unexpected token u - folders length - 41","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"there was a problem processing i=4(Calibre Library) - SyntaxError: Unexpected token u - folders length - 42","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=5(My Music) - SyntaxError: Unexpected token u - folders length - 43","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=6(TurboTax) - SyntaxError: Unexpected token u - folders length - 44","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=7(My Pictures) - SyntaxError: Unexpected token u - folders length - 45","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=8(My Videos) - SyntaxError: Unexpected token u - folders length - 46","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=9(Outlook Files) - SyntaxError: Unexpected token u - folders length - 47","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=10(Virtual Machines) - SyntaxError: Unexpected token u - folders length - 48","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=11(WBFS Manager Covers) - SyntaxError: Unexpected token u - folders length - 49","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=12(Camtasia Studio) - SyntaxError: Unexpected token u - folders length - 50","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=13(OneNote Notebooks) - SyntaxError: Unexpected token u - folders length - 51","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=14(Custom Office Templates) - SyntaxError: Unexpected token u - folders length - 52","timestamp":"2014-08-06T16:12:31.153Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.153Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:12:38.522Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:17:54.984Z"} {"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token u - folders length - 38","timestamp":"2014-08-06T16:23:04.058Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"there was a problem processing i=1(SuperPuTTY) - SyntaxError: Unexpected token u - folders length - 39","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"there was a problem processing i=2(MyJabberFiles) - SyntaxError: Unexpected token u - folders length - 40","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"there was a problem processing i=3(cache) - SyntaxError: Unexpected token u - folders length - 41","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"there was a problem processing i=4(Calibre Library) - SyntaxError: Unexpected token u - folders length - 42","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"there was a problem processing i=5(My Music) - SyntaxError: Unexpected token u - folders length - 43","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"there was a problem processing i=6(TurboTax) - SyntaxError: Unexpected token u - folders length - 44","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=7(My Pictures) - SyntaxError: Unexpected token u - folders length - 45","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=8(My Videos) - SyntaxError: Unexpected token u - folders length - 46","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=9(Outlook Files) - SyntaxError: Unexpected token u - folders length - 47","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=10(Virtual Machines) - SyntaxError: Unexpected token u - folders length - 48","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=11(WBFS Manager Covers) - SyntaxError: Unexpected token u - folders length - 49","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"there was a problem processing i=12(Camtasia Studio) - SyntaxError: Unexpected token u - folders length - 50","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"there was a problem processing i=13(OneNote Notebooks) - SyntaxError: Unexpected token u - folders length - 51","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"there was a problem processing i=14(Custom Office Templates) - SyntaxError: Unexpected token u - folders length - 52","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"there was a problem processing i=15(Files) - SyntaxError: Unexpected token u - folders length - 53","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"there was a problem processing i=16(TitantiumBackup) - SyntaxError: Unexpected token u - folders length - 54","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"there was a problem processing i=17(Media) - SyntaxError: Unexpected token u - folders length - 55","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"there was a problem processing i=18(MP3s) - SyntaxError: Unexpected token u - folders length - 56","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"there was a problem processing i=19(Pictures) - SyntaxError: Unexpected token u - folders length - 57","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"there was a problem processing i=20(System Volume Information) - SyntaxError: Unexpected token u - folders length - 58","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"there was a problem processing i=21(iTunes) - SyntaxError: Unexpected token u - folders length - 59","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"there was a problem processing i=22(Google Music) - SyntaxError: Unexpected token u - folders length - 60","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"there was a problem processing i=23(/) - SyntaxError: Unexpected token u - folders length - 61","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"there was a problem processing i=24(HelloFax) - SyntaxError: Unexpected token u - folders length - 62","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"there was a problem processing i=25(sakhter.net) - SyntaxError: Unexpected token u - folders length - 63","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"there was a problem processing i=26(Random) - SyntaxError: Unexpected token u - folders length - 64","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"there was a problem processing i=27(Important Docs) - SyntaxError: Unexpected token u - folders length - 65","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"error","message":"there was a problem processing i=28(scripts) - SyntaxError: Unexpected token u - folders length - 66","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"error","message":"there was a problem processing i=29(Google Music) - SyntaxError: Unexpected token u - folders length - 67","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:23:18.972Z"} {"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token u - folders length - 38","timestamp":"2014-08-06T16:28:28.055Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.056Z"} {"level":"error","message":"there was a problem processing i=1(SuperPuTTY) - SyntaxError: Unexpected token u - folders length - 39","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"there was a problem processing i=2(MyJabberFiles) - SyntaxError: Unexpected token u - folders length - 40","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"there was a problem processing i=3(cache) - SyntaxError: Unexpected token u - folders length - 41","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"there was a problem processing i=4(Calibre Library) - SyntaxError: Unexpected token u - folders length - 42","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"there was a problem processing i=5(My Music) - SyntaxError: Unexpected token u - folders length - 43","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"there was a problem processing i=6(TurboTax) - SyntaxError: Unexpected token u - folders length - 44","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"there was a problem processing i=7(My Pictures) - SyntaxError: Unexpected token u - folders length - 45","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"there was a problem processing i=8(My Videos) - SyntaxError: Unexpected token u - folders length - 46","timestamp":"2014-08-06T16:28:28.059Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.059Z"} {"level":"error","message":"there was a problem processing i=9(Outlook Files) - SyntaxError: Unexpected token u - folders length - 47","timestamp":"2014-08-06T16:28:28.059Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.060Z"} {"level":"error","message":"there was a problem processing i=10(Virtual Machines) - SyntaxError: Unexpected token u - folders length - 48","timestamp":"2014-08-06T16:28:28.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.060Z"} {"level":"error","message":"there was a problem processing i=11(WBFS Manager Covers) - SyntaxError: Unexpected token u - folders length - 49","timestamp":"2014-08-06T16:28:28.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"there was a problem processing i=12(Camtasia Studio) - SyntaxError: Unexpected token u - folders length - 50","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"there was a problem processing i=13(OneNote Notebooks) - SyntaxError: Unexpected token u - folders length - 51","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"there was a problem processing i=14(Custom Office Templates) - SyntaxError: Unexpected token u - folders length - 52","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"there was a problem processing i=15(Files) - SyntaxError: Unexpected token u - folders length - 53","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"there was a problem processing i=16(TitantiumBackup) - SyntaxError: Unexpected token u - folders length - 54","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"there was a problem processing i=17(Media) - SyntaxError: Unexpected token u - folders length - 55","timestamp":"2014-08-06T16:28:28.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.064Z"} {"level":"error","message":"there was a problem processing i=18(MP3s) - SyntaxError: Unexpected token u - folders length - 56","timestamp":"2014-08-06T16:28:28.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.064Z"} {"level":"error","message":"there was a problem processing i=19(Pictures) - SyntaxError: Unexpected token u - folders length - 57","timestamp":"2014-08-06T16:28:28.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"there was a problem processing i=20(System Volume Information) - SyntaxError: Unexpected token u - folders length - 58","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"there was a problem processing i=21(iTunes) - SyntaxError: Unexpected token u - folders length - 59","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"there was a problem processing i=22(Google Music) - SyntaxError: Unexpected token u - folders length - 60","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:28:44.882Z"}

I noticed /mnt/BitcasaJS/Bitcasa Infinite Drive/Media was in the error list....

I then went to that folder and did an LS:

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T16:32:21.723Z"}

After 2-3 minutes:

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls Indian_Movies Movies Movies2 TVShows TVShows_Finished TVShows_Other sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

Shortly afterwards:

{"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:33:58.132Z"}

No errors...

and now:

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

is empty...

[saqebakhter]

If you want access to my machine your more than welcome to debug!

[EricTheMagician]

You said you edited the source code: The errors you are getting are related to it. They all say "json undefined".

I haven't had that error before. Even in your previous logs, it didn't have that.

Eric

On Wed, Aug 6, 2014 at 12:35 PM, saqebakhter notifications@github.com wrote:

I changed the refresh time to 5 minutes to make this go faster - i also tried to print the json - data variable thats what the - "json - undefined", not sure why it didn't print.

Went to /mnt/BitcasaFS - did an LS then waited:

{"level":"debug","message":"getting folder info from bitcasa for / -- depth 1","timestamp":"2014-08-06T16:07:11.302Z"} {"level":"info","message":"mount point: /mnt/BitcasaJS","timestamp":"2014-08-06T16:07:11.321Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:07:22.055Z"}

{"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token u - folders length - 38","timestamp":"2014-08-06T16:12:31.148Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.149Z"} {"level":"error","message":"there was a problem processing i=1(SuperPuTTY)

• SyntaxError: Unexpected token u - folders length - 39","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"there was a problem processing i=2(MyJabberFiles) - SyntaxError: Unexpected token u - folders length - 40","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"there was a problem processing i=3(cache) - SyntaxError: Unexpected token u - folders length - 41","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.150Z"} {"level":"error","message":"there was a problem processing i=4(Calibre Library) - SyntaxError: Unexpected token u - folders length - 42","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=5(My Music) - SyntaxError: Unexpected token u - folders length - 43","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=6(TurboTax) - SyntaxError: Unexpected token u - folders length - 44","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=7(My Pictures) - SyntaxError: Unexpected token u - folders length - 45","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=8(My Videos)
• SyntaxError: Unexpected token u - folders length - 46","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.151Z"} {"level":"error","message":"there was a problem processing i=9(Outlook Files) - SyntaxError: Unexpected token u - folders length - 47","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=10(Virtual Machines) - SyntaxError: Unexpected token u - folders length - 48","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=11(WBFS Manager Covers) - SyntaxError: Unexpected token u - folders length - 49","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=12(Camtasia Studio) - SyntaxError: Unexpected token u - folders length - 50","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=13(OneNote Notebooks) - SyntaxError: Unexpected token u - folders length - 51","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.152Z"} {"level":"error","message":"there was a problem processing i=14(Custom Office Templates) - SyntaxError: Unexpected token u - folders length - 52","timestamp":"2014-08-06T16:12:31.153Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:12:31.153Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:12:38.522Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:17:54.984Z"} {"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token u - folders length - 38","timestamp":"2014-08-06T16:23:04.058Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"there was a problem processing i=1(SuperPuTTY)
• SyntaxError: Unexpected token u - folders length - 39","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"there was a problem processing i=2(MyJabberFiles) - SyntaxError: Unexpected token u - folders length - 40","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.059Z"} {"level":"error","message":"there was a problem processing i=3(cache) - SyntaxError: Unexpected token u - folders length - 41","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"there was a problem processing i=4(Calibre Library) - SyntaxError: Unexpected token u - folders length - 42","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"there was a problem processing i=5(My Music) - SyntaxError: Unexpected token u - folders length - 43","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"there was a problem processing i=6(TurboTax) - SyntaxError: Unexpected token u - folders length - 44","timestamp":"2014-08-06T16:23:04.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=7(My Pictures) - SyntaxError: Unexpected token u - folders length - 45","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=8(My Videos)
• SyntaxError: Unexpected token u - folders length - 46","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=9(Outlook Files) - SyntaxError: Unexpected token u - folders length - 47","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=10(Virtual Machines) - SyntaxError: Unexpected token u - folders length - 48","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.061Z"} {"level":"error","message":"there was a problem processing i=11(WBFS Manager Covers) - SyntaxError: Unexpected token u - folders length - 49","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"there was a problem processing i=12(Camtasia Studio) - SyntaxError: Unexpected token u - folders length - 50","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"there was a problem processing i=13(OneNote Notebooks) - SyntaxError: Unexpected token u - folders length - 51","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"there was a problem processing i=14(Custom Office Templates) - SyntaxError: Unexpected token u - folders length - 52","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.062Z"} {"level":"error","message":"there was a problem processing i=15(Files) - SyntaxError: Unexpected token u - folders length - 53","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"there was a problem processing i=16(TitantiumBackup) - SyntaxError: Unexpected token u - folders length - 54","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"there was a problem processing i=17(Media) - SyntaxError: Unexpected token u - folders length - 55","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"there was a problem processing i=18(MP3s) - SyntaxError: Unexpected token u - folders length - 56","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"there was a problem processing i=19(Pictures)
• SyntaxError: Unexpected token u - folders length - 57","timestamp":"2014-08-06T16:23:04.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"there was a problem processing i=20(System Volume Information) - SyntaxError: Unexpected token u - folders length - 58","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"there was a problem processing i=21(iTunes) - SyntaxError: Unexpected token u - folders length - 59","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"there was a problem processing i=22(Google Music) - SyntaxError: Unexpected token u - folders length - 60","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"there was a problem processing i=23(/) - SyntaxError: Unexpected token u - folders length - 61","timestamp":"2014-08-06T16:23:04.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"there was a problem processing i=24(HelloFax)
• SyntaxError: Unexpected token u - folders length - 62","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"there was a problem processing i=25( sakhter.net) - SyntaxError: Unexpected token u - folders length - 63","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"there was a problem processing i=26(Random) - SyntaxError: Unexpected token u - folders length - 64","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"there was a problem processing i=27(Important Docs) - SyntaxError: Unexpected token u - folders length - 65","timestamp":"2014-08-06T16:23:04.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"error","message":"there was a problem processing i=28(scripts) - SyntaxError: Unexpected token u - folders length - 66","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"error","message":"there was a problem processing i=29(Google Music) - SyntaxError: Unexpected token u - folders length - 67","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:23:04.066Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:23:18.972Z"} {"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token u - folders length - 38","timestamp":"2014-08-06T16:28:28.055Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.056Z"} {"level":"error","message":"there was a problem processing i=1(SuperPuTTY)
• SyntaxError: Unexpected token u - folders length - 39","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"there was a problem processing i=2(MyJabberFiles) - SyntaxError: Unexpected token u - folders length - 40","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"there was a problem processing i=3(cache) - SyntaxError: Unexpected token u - folders length - 41","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"there was a problem processing i=4(Calibre Library) - SyntaxError: Unexpected token u - folders length - 42","timestamp":"2014-08-06T16:28:28.057Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"there was a problem processing i=5(My Music) - SyntaxError: Unexpected token u - folders length - 43","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"there was a problem processing i=6(TurboTax) - SyntaxError: Unexpected token u - folders length - 44","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"there was a problem processing i=7(My Pictures) - SyntaxError: Unexpected token u - folders length - 45","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.058Z"} {"level":"error","message":"there was a problem processing i=8(My Videos)
• SyntaxError: Unexpected token u - folders length - 46","timestamp":"2014-08-06T16:28:28.059Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.059Z"} {"level":"error","message":"there was a problem processing i=9(Outlook Files) - SyntaxError: Unexpected token u - folders length - 47","timestamp":"2014-08-06T16:28:28.059Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.060Z"} {"level":"error","message":"there was a problem processing i=10(Virtual Machines) - SyntaxError: Unexpected token u - folders length - 48","timestamp":"2014-08-06T16:28:28.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.060Z"} {"level":"error","message":"there was a problem processing i=11(WBFS Manager Covers) - SyntaxError: Unexpected token u - folders length - 49","timestamp":"2014-08-06T16:28:28.060Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"there was a problem processing i=12(Camtasia Studio) - SyntaxError: Unexpected token u - folders length - 50","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"there was a problem processing i=13(OneNote Notebooks) - SyntaxError: Unexpected token u - folders length - 51","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"there was a problem processing i=14(Custom Office Templates) - SyntaxError: Unexpected token u - folders length - 52","timestamp":"2014-08-06T16:28:28.061Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"there was a problem processing i=15(Files) - SyntaxError: Unexpected token u - folders length - 53","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"there was a problem processing i=16(TitantiumBackup) - SyntaxError: Unexpected token u - folders length - 54","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.062Z"} {"level":"error","message":"there was a problem processing i=17(Media) - SyntaxError: Unexpected token u - folders length - 55","timestamp":"2014-08-06T16:28:28.063Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.064Z"} {"level":"error","message":"there was a problem processing i=18(MP3s) - SyntaxError: Unexpected token u - folders length - 56","timestamp":"2014-08-06T16:28:28.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.064Z"} {"level":"error","message":"there was a problem processing i=19(Pictures)
• SyntaxError: Unexpected token u - folders length - 57","timestamp":"2014-08-06T16:28:28.064Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"there was a problem processing i=20(System Volume Information) - SyntaxError: Unexpected token u - folders length - 58","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"there was a problem processing i=21(iTunes) - SyntaxError: Unexpected token u - folders length - 59","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"there was a problem processing i=22(Google Music) - SyntaxError: Unexpected token u - folders length - 60","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"error","message":"json - undefined","timestamp":"2014-08-06T16:28:28.065Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:28:44.882Z"}

I noticed /mnt/BitcasaJS/Bitcasa Infinite Drive/Media was in the error list....

I then went to that folder and did an LS:

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T16:32:21.723Z"}

After 2-3 minutes:

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls Indian_Movies Movies Movies2 TVShows TVShows_Finished TVShows_Other sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

Shortly afterwards:

{"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T16:33:58.132Z"}

No errors...

and now:

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

is empty...

— Reply to this email directly or view it on GitHub https://github.com/thejinx0r/BitcasaF4JS/issues/3#issuecomment-51361186.

[saqebakhter]

Yah i adddd that to print the json text - but i dont thin it worked just a console.log

    catch error
      folders.push(folders[i])
      client.logger.log "error", "there was a problem processing i=#{i}(#{folders[i].name}) - #{error} - folders length - #{folders.length}"
      client.logger.log "error", "json - #{data}"
      continue

[saqebakhter]

Okay - the only thing i added was a console.log so it should not impact the rest of the code , the issue with /Media losing its structure is still true though.

[EricTheMagician]

Can you provide a new log with out changes to the code? It does appear as though you have not defined a variable properly.

— Sent from Mailbox

On Wed, Aug 6, 2014 at 1:35 PM, saqebakhter notifications@github.com wrote:

Okay - the only thing i added was a console.log so it should not impact the rest of the code , the issue with /Media losing its structure is still true though.

Reply to this email directly or view it on GitHub: https://github.com/thejinx0r/BitcasaF4JS/issues/3#issuecomment-51368978

[saqebakhter]

Yes - i'll do a git reset and follow the same steps. It will take longer though as you have it set to a 15 minute refresh for the folders.

Is there a way to change to a finer logging within the the current code ?

[EricTheMagician]

It's fine, you don't need to do it. The block code you put didn't show up on my phone.

[EricTheMagician]

Also, which commit are you on?

[saqebakhter]

I just did a git pull / git reset - i'm currently on ,

commit 8f3ed94dd5781893d9492cb5087219249a7f899c -use setInterval instead of setTimeout

I'm redoing the test right now. I think there is some API calls that are failing but are not being detected - and the console.log I had was correct it was just a null value because the API call failed. I'm not very famillar with node.js still getting used to it ( python/c ) is where I'm better :)

[saqebakhter]

Okay here it is again:

Started up bitcasaFS

{"level":"debug","message":"getting folder info from bitcasa for / -- depth 1","timestamp":"2014-08-06T18:11:09.764Z"} {"level":"info","message":"mount point: /mnt/BitcasaJS","timestamp":"2014-08-06T18:11:09.783Z"} {"level":"info","message":"Starting fuse4js on BitcasaFuse4JS","timestamp":"2014-08-06T18:11:09.789Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T18:11:14.659Z"}

cd to Media folder

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive$ cd Media sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls -lrt total 0 sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ pwd /mnt/BitcasaJS/Bitcasa Infinite Drive/Media sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

This triggers a refresh on that folder:

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T18:11:44.630Z"}

Few minutes later that folder has children:

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T18:11:44.630Z"}

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls -lrt total 0 drwxrwxrwx 35 root root 4096 Jun 18 11:26 TVShows_Finished drwxrwxrwx 943 root root 4096 Aug 3 13:26 Movies drwxrwxrwx 51 root root 4096 Aug 3 19:43 TVShows drwxrwxrwx 42 root root 4096 Aug 3 20:11 TVShows_Other drwxrwxrwx 59 root root 4096 Aug 4 11:51 Movies2 sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

15 minutes later the getAllFolders ran:

{"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token u - folders length - 38","timestamp":"2014-08-06T18:26:18.849Z"} {"level":"error","message":"there was a problem processing i=1(SuperPuTTY) - SyntaxError: Unexpected token u - folders length - 39","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=2(MyJabberFiles) - SyntaxError: Unexpected token u - folders length - 40","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=3(cache) - SyntaxError: Unexpected token u - folders length - 41","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=4(Calibre Library) - SyntaxError: Unexpected token u - folders length - 42","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=5(My Music) - SyntaxError: Unexpected token u - folders length - 43","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=6(TurboTax) - SyntaxError: Unexpected token u - folders length - 44","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=7(My Pictures) - SyntaxError: Unexpected token u - folders length - 45","timestamp":"2014-08-06T18:26:18.851Z"} {"level":"error","message":"there was a problem processing i=8(My Videos) - SyntaxError: Unexpected token u - folders length - 46","timestamp":"2014-08-06T18:26:18.851Z"} {"level":"error","message":"there was a problem processing i=9(Outlook Files) - SyntaxError: Unexpected token u - folders length - 47","timestamp":"2014-08-06T18:26:18.851Z"} {"level":"error","message":"there was a problem processing i=10(Virtual Machines) - SyntaxError: Unexpected token u - folders length - 48","timestamp":"2014-08-06T18:26:18.851Z"} {"level":"error","message":"there was a problem processing i=11(WBFS Manager Covers) - SyntaxError: Unexpected token u - folders length - 49","timestamp":"2014-08-06T18:26:18.852Z"} {"level":"error","message":"there was a problem processing i=12(Camtasia Studio) - SyntaxError: Unexpected token u - folders length - 50","timestamp":"2014-08-06T18:26:18.852Z"} {"level":"error","message":"there was a problem processing i=13(OneNote Notebooks) - SyntaxError: Unexpected token u - folders length - 51","timestamp":"2014-08-06T18:26:18.852Z"} {"level":"error","message":"there was a problem processing i=14(Custom Office Templates) - SyntaxError: Unexpected token u - folders length - 52","timestamp":"2014-08-06T18:26:18.852Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T18:26:44.734Z"}

Lots of errors....

Now if i ls -lrt on Media folder:

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls -lrt total 0 sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

Its empty , and triggers a refresh again in bitcasaFS

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T18:27:36.521Z"}

[EricTheMagician]

The only thing that I can think of is that your Infinite Drive has so many files that Bitcasa is having troubles returning them without killing the connections.

Eric

On Wed, Aug 6, 2014 at 2:28 PM, saqebakhter notifications@github.com wrote:

Okay here it is again:

Started up bitcasaFS

{"level":"debug","message":"getting folder info from bitcasa for / -- depth 1","timestamp":"2014-08-06T18:11:09.764Z"} {"level":"info","message":"mount point: /mnt/BitcasaJS","timestamp":"2014-08-06T18:11:09.783Z"} {"level":"info","message":"Starting fuse4js on BitcasaFuse4JS","timestamp":"2014-08-06T18:11:09.789Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T18:11:14.659Z"}

cd to Media folder

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive$ cd Media sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls -lrt total 0 sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ pwd /mnt/BitcasaJS/Bitcasa Infinite Drive/Media sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

This triggers a refresh on that folder:

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T18:11:44.630Z"}

Few minutes later that folder has children:

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T18:11:44.630Z"}

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls -lrt total 0 drwxrwxrwx 35 root root 4096 Jun 18 11:26 TVShows_Finished drwxrwxrwx 943 root root 4096 Aug 3 13:26 Movies drwxrwxrwx 51 root root 4096 Aug 3 19:43 TVShows drwxrwxrwx 42 root root 4096 Aug 3 20:11 TVShows_Other drwxrwxrwx 59 root root 4096 Aug 4 11:51 Movies2 sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

15 minutes later the getAllFolders ran:

{"level":"error","message":"there was a problem processing i=0(root) - SyntaxError: Unexpected token u - folders length - 38","timestamp":"2014-08-06T18:26:18.849Z"} {"level":"error","message":"there was a problem processing i=1(SuperPuTTY)

• SyntaxError: Unexpected token u - folders length - 39","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=2(MyJabberFiles) - SyntaxError: Unexpected token u - folders length - 40","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=3(cache) - SyntaxError: Unexpected token u - folders length - 41","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=4(Calibre Library) - SyntaxError: Unexpected token u - folders length - 42","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=5(My Music) - SyntaxError: Unexpected token u - folders length - 43","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=6(TurboTax) - SyntaxError: Unexpected token u - folders length - 44","timestamp":"2014-08-06T18:26:18.850Z"} {"level":"error","message":"there was a problem processing i=7(My Pictures) - SyntaxError: Unexpected token u - folders length - 45","timestamp":"2014-08-06T18:26:18.851Z"} {"level":"error","message":"there was a problem processing i=8(My Videos)
• SyntaxError: Unexpected token u - folders length - 46","timestamp":"2014-08-06T18:26:18.851Z"} {"level":"error","message":"there was a problem processing i=9(Outlook Files) - SyntaxError: Unexpected token u - folders length - 47","timestamp":"2014-08-06T18:26:18.851Z"} {"level":"error","message":"there was a problem processing i=10(Virtual Machines) - SyntaxError: Unexpected token u - folders length - 48","timestamp":"2014-08-06T18:26:18.851Z"} {"level":"error","message":"there was a problem processing i=11(WBFS Manager Covers) - SyntaxError: Unexpected token u - folders length - 49","timestamp":"2014-08-06T18:26:18.852Z"} {"level":"error","message":"there was a problem processing i=12(Camtasia Studio) - SyntaxError: Unexpected token u - folders length - 50","timestamp":"2014-08-06T18:26:18.852Z"} {"level":"error","message":"there was a problem processing i=13(OneNote Notebooks) - SyntaxError: Unexpected token u - folders length - 51","timestamp":"2014-08-06T18:26:18.852Z"} {"level":"error","message":"there was a problem processing i=14(Custom Office Templates) - SyntaxError: Unexpected token u - folders length - 52","timestamp":"2014-08-06T18:26:18.852Z"} {"level":"debug","message":"it took 1 minutes to update folders","timestamp":"2014-08-06T18:26:44.734Z"}

Lots of errors....

Now if i ls -lrt on Media folder:

sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$ ls -lrt total 0 sakhter@sakhter-ubuntu:/mnt/BitcasaJS/Bitcasa Infinite Drive/Media$

Its empty , and triggers a refresh again in bitcasaFS

{"level":"debug","message":"getting folder info from bitcasa for /Bitcasa Infinite Drive/Media -- depth 3","timestamp":"2014-08-06T18:27:36.521Z"}

— Reply to this email directly or view it on GitHub https://github.com/thejinx0r/BitcasaF4JS/issues/3#issuecomment-51376291.

[saqebakhter]

Quite possible. I have 14Tb of Media.

I'll see if i can debug this some more.

[saqebakhter]

I think i found the issue in fs.coffee in this code block:

while processing.length < folders.length
        client.logger.log  "silly", "folders length = #{folders.length}, processing length: #{processing.length}"
        tokens = Math.min(Math.ceil(client.rateLimit.getTokensRemaining()/12), folders.length - processing.length)
        for i in [0...tokens]
          if not client.rateLimit.tryRemoveTokens(1)
            setTimeout fiberRun, 1000
            Fiber.yield()
          depth = 3
          if folders[i].bitcasaPath == '/'
            depth = 1
          processing.push getFolder(folders[i].bitcasaPath,depth)
        if processing.length < folders.length
          setTimeout fiberRun, 4500
          Fiber.yield()

You are looping through , and adding the folders to the process queue - BUT if you don't have enough tokens ( which I usually don't ) you are adding the same folders to the process queue over and over again becaues the for loop for i in [0...tokens], selects the same folders, and pushes them into the process queue.

[EricTheMagician]

Please use the code blocks next time, because the code is actually in coffeescript, which compiles to javascript. In coffeescript, the indentation matters, like in python. So instead of {} in most C-like languages, coffeescript uses indentations.

It doesn't keep trying to add them over and over again. Once they have been processed, I removed them from the folders array. It's the folders.splice that further down

But that's not the problem. The problem has to do with the data being undefined that showed up in your logs. I forgot an else statement in this code block to set data.

        if processing[i].isResolved()
          data = processing[i].get()
        else
          #if processing is not done, wait 60 seconds.
          setTimeout fiberRun, 60000
          Fiber.yield()
          try #catch socket connection error
            client.logger.log "silly", "waiting for process[#{i}]. folder name is: #{folders[i].name}"
            processing[i].wait()
            data = processing[i].get()
          catch error
            client.logger.log("error", "there was a problem with connection for folder #{folders[i].name} - #{error}")
            folders.push(folders[i])
            continue

More errors popped up, when I fix those. So I will push the commits once those are sorted.

[EricTheMagician]

It has been pushed to master.

[saqebakhter]

Looks like its much better but I think you left the refresh interval too fast:

setTimeout getAllFolders, 30000

[EricTheMagician]

Currently, it takes about 45 minutes on my system to do the update. So the interval refresh is fine to me.

— Sent from Mailbox

On Thu, Aug 7, 2014 at 10:41 AM, saqebakhter notifications@github.com wrote:

Looks like its much better but I think you left the refresh interval too fast:

setTimeout getAllFolders, 30000

Reply to this email directly or view it on GitHub: https://github.com/thejinx0r/BitcasaF4JS/issues/3#issuecomment-51480937

[saqebakhter]

Yes - I realized its taking lot longer now. Too bad there API doesn't have a better mechanism. When you contacted support did they mention they are working on anything to make this better ?

[EricTheMagician]

They do have a better mechanism. In theory, you can get all files and folders with one API call. The limitation is that all API calls must terminate in under a minute if not, the connection is terminated. When I asked about potentially increasing it, they said that their dev team will look in to it, but it will likely not happen any time soon.

Also folder saving/loading seems to be working, I will merge it to master soon. Right now, I'm just looking in to the performance of the filesystem.