Folder Navigation Problem

[GoogleCodeExporter]

I'm not sure if this is a bug or not, but i attached a picture to explain what 
is going on.
On the picture you can see that a user has navigated deep into few folders, but 
can't get back to the previous folder by clicking on the previous folder name, 
since they are tied together as one link and by clicking on it, nothing 
happens, user stays in the same folder where he/she currently is.

I'm guessing this is some sort of windows issue again, but you never know.

Also it would be great if there would be "Back" and "Forward" buttons to 
navigate the folders back and forth. Most users can't figure out you can just 
simply click on the folder name and get to the place you want to be, so these 
arrow buttons would be great.

Original issue reported on code.google.com by Logic...@gmail.com on 2 Nov 2013 at 11:23

Attachments:

• Folder_Path.png

[GoogleCodeExporter]

Definitely windows problem. I'm gonna dig out the windows virtual machine and 
take a look at this (and the charset)

Original comment by samuli.j...@gmail.com on 3 Nov 2013 at 7:57

• Changed state: Accepted

[GoogleCodeExporter]

I've installed Windows 7 virtual machine with XAMP server, and made some 
changes. At least in my environment I am now able to navigate on different 
levels and handle files (including upload) with chars like "ñäöå"

I've uploaded this version here: 
https://code.google.com/p/mollify/downloads/detail?name=mollify_2.2.9b1.zip

It is marked as beta because the changes I made were quite big, and I need to 
test it more before I release it. But if you could test it, it would be great.

I was using configuration setting
"convert_filenames" => "CP1252"

Original comment by samuli.j...@gmail.com on 6 Nov 2013 at 10:28

[GoogleCodeExporter]

This is pretty much incredible what you have done, but i still have few issues.

Namely i have a file called: (testing with that file seems to bring out all 
sort of interesting issues everywhere) "Enya - La Soñadora.mp3". When i upload 
that file (thanks to your update, i now CAN upload that file) it changes the 
special letters to, as follows: "Enya - La Sońadora.mp3", notice the "n" 
letter has changed. It uploads correctly and it can be viewed after upload and 
even be used (downloaded etc) also the name viewed FROM Mollify displays the 
name correctly "Enya - La Soñadora.mp3", BUT on the file system, viewed from 
regular windows explorer, is on the disk with the name: "Enya - La 
Sońadora.mp3".

This now results another issue that if i put the file "Enya - La Soñadora.mp3" 
directly on the Mollifys user path on the local disk with simple copy/paste 
action, then Mollify cannot read it anymore and results a "Request Failed". 
Also it adds to PHP Error Log (debug disabled, tell me if you need the whole 
shebang): 

MOLLIFY ERROR: PHP error #2, is_dir(): open_basedir restriction in effect. 
File(V:\Local_Directory_on_Disk2\Mollify\blabla\Test\Enya - La So?adora.Mp3) is 
not within the allowed path(s)...

I'm guessing the "?" breaks the configuration readability for PHP itself.
I haven't tested by changing the Apache charset to anything different than 
UTF-8 because i would still need to have UTF-8 enabled in Apache. Also PHP is 
set to have UTF-8 enabled for my other scripts that need/might need it, not 
mentioning MySQL, they all have UTF-8.

Before i tested i DID put the:
            "convert_filenames" => "CP1252",
and i had "charset on:
            "charset" => "utf8",

So basically reading the files from disk which haven't been put there VIA 
Mollify, is still an issue, at least for me and that is the major problem here.

One more positive note is that the folder navigation now works correctly.

Please let me know if you need more information.

Original comment by Logic...@gmail.com on 7 Nov 2013 at 6:56

[GoogleCodeExporter]

It is clear that the conversion is not correct yet. Whenever some char ends up 
different, there is wrong conversion done. And of course it is natural that 
when even one char is converted incorrectly, the file is not found. It's the 
same as replacing all "a"'s into "b", the files simply don't exist with those 
names.

The question is, what is the correct conversion. Are you sure it is supposed to 
be "CP1252"? I'm no expert on a) windows or b) charsets in general, and cannot 
identify what is the charset that has "ń" where CP1252 has "ñ".

Basically you would have to create a test php page that lists the files in this 
particular folder, and makes a conversion. There you could try out different 
encodings to see which one produces the correct utf-8 result. I don't have this 
kind of test page, don't know if I had time to make one...

The most important thing here is that, _everything_ must operate on utf-8. PHP, 
Apache, MySQL and website. Mollify assumes that all strings are utf-8. And only 
in case of windows, the last layer must be converted, which is the filesystem.

I think my own server works as a proof of concept that conversion can be done, 
since for example that exact filename "Enya - La Soñadora.mp3" works, both 
when entered directly into windows filesystem, or entered via Mollify app.

btw, your error is again the "open_basedir", same as with the hashing problem. 
You removed the beginning of the path (for privacy reasons, of course), did you 
check that the rest of the path was actually correct (excluding the ? mark in 
filename)?

Original comment by samuli.j...@gmail.com on 7 Nov 2013 at 12:46

[GoogleCodeExporter]

OK, so I made the test script, didn't seem too complex. Just copy it into your 
web server, edit it and modify the variables. Try the folder where you have the 
problematic file, and see what the browser says.

Now this script does make the same conversion as mollify, so you should see the 
file names exactly as they are in filesystem. In my windows server, CP1252 does 
output the letters correctly, including "ñ".

Original comment by samuli.j...@gmail.com on 7 Nov 2013 at 1:05

Attachments:

• charset.php

[GoogleCodeExporter]

This is driving me insane.
Can you send me your configuration files on PHP, Apache and MySQL, also the 
version numbers and are they for 32bit or 64bit architecture? Also what version 
of Windows are you running this on and what are the regional and language 
settings?

I tried a lot of charsets and the closest i got was "CP1257" which did display 
some symbols what "CP1252" didn't show, but STILL, can't encode correctly the 
"ñ".

Since it's working for you and not for me, I'm guessing there has to be 
something wrong with my configuration somewhere in Apache, PHP, MySQL or 
Windows in general.

Original comment by Logic...@gmail.com on 7 Nov 2013 at 6:55

[GoogleCodeExporter]

Did some more testing and it seems it also can't translate random russian 
letters, for example "фртыйкмвхжэщъыяёсёё" and replaces them 
with "?" marks. I'm assuming it can't do half the stuff in the world and i 
can't figure out what conversion must i use. Also it doesn't make sense your 
conversion charset works, because in that charset "CP1252" there are no russian 
letters...

Something else in here is fishy

Original comment by Logic...@gmail.com on 7 Nov 2013 at 7:11

[GoogleCodeExporter]

Not sure if this helps, but i changed the settings in your script from "CP..." 
to UTF-8 just randomly to test some things and then i got an error log not from 
Mollify, but from PHP itself with:

PHP Notice:  iconv(): Detected an illegal character in input string in 
C:\PATH_TO_MOLLIFY\charset.php on line 16

I'm not sure what this is suppose to mean, as i didn't change the line 16, but 
the line 4.

Original comment by Logic...@gmail.com on 7 Nov 2013 at 7:18

[GoogleCodeExporter]

I installed WAMP server myself on a Virtual Machine and i STILL have the same 
problems in there, i tested only with your charset script.
I didn't modify any other settings and used WAMP servers default settings. How 
the hell is that even possible now. I'm going insane.
It simply works for you and not for me.

Original comment by Logic...@gmail.com on 7 Nov 2013 at 8:12

[GoogleCodeExporter]

I got my friend to help test this out as well, i just got a random picture and 
named it to: 
"Sońa_Soña_Süõäö_€tere_фртыйкмвхжэщъыяёсёё_.jpg" 
just as a test input string to see how it works. He didn't get it to work under 
Windows server as well, under windows server it shows: 
"Soña_So?a_Süõäö_€tere_???????????????????_.jpg" with the settings 
CP1252. He got it working under linux environment while setting the encoding to 
UTF-8, but it's clear anyway that it works under Linux. Also i set up, as i 
said before, a VM with default installation of WAMP and didn't get it to work 
either, the results were the same. So i am very curious what server 
configuration you have that it works in your environment.

Original comment by Logic...@gmail.com on 7 Nov 2013 at 11:25

[GoogleCodeExporter]

Your example with russian characters did not work either on my server, the last 
part is all question marks. So my charset is not CP1252 either, and from 
reading posts it seems it should be UTF-16 in NTFS filesystem. But this did not 
work either, not even close. So I can say that this really is unnecessary 
difficult with microsoft products, both windows and IE cause so many extra 
troubles, I wish I could just ignore them...

For example this page http://evertpot.com/filesystem-encoding-and-php/ has a 
conclusion "It would be possible to provide a consistent experience, if it 
weren't for windows. Windows does have all the proper api's to deal with 
international filenames, but I can only assume PHP simply does not support 
them. I do believe this was scheduled for PHP6, but now that's off the hook. I 
hope the filesystem api's are replaced even before the entire language is 
unicode-based."

Original comment by samuli.j...@gmail.com on 8 Nov 2013 at 6:03

[GoogleCodeExporter]

I too found the UTF-16 and it also did not work, i even tried loads of other 
ridiculous solutions and of course, no help.
My friend also told me that this might be literally PHP issue itself.
I was waiting for your response and now I'm sad.
I set back the charset encoding to UTF-8 so at least people can upload anything 
to the server, but i can't implement half my ideas i wish i could.
I guess we just have to wait for PHP6 release.

Either way, thank you so much for putting your time and effort in it, it was 
hell, but at least now we know the answer.

Original comment by Logic...@gmail.com on 8 Nov 2013 at 6:27

[GoogleCodeExporter]

One last trial I'm gonna make. According to this post 
http://stackoverflow.com/questions/1525830/how-do-i-use-filesystem-functions-in-
php-using-utf-8-strings it should be possible to just urlencode the utf-8 
filenames, which the filesystem should accept and interpret according to own 
charset.

I have a doubt that if you just urlencode utf-8 char, it will simply transform 
any non-ascii letter into its byte values, for example "ñ" -> "%C3%B1". These 
values are utf-8 byte values in hex, and I'm really wondering that how can this 
work regardless of target charset? I mean is the C3+B1 always "ñ" in any 
windows filesystem no matter if it is in chinese locale etc?

But we'll see, I let you know how it turns out.

Original comment by samuli.j...@gmail.com on 8 Nov 2013 at 10:02

[GoogleCodeExporter]

Oh no, just realizes after pressing submit, that this is for entirely different 
goal.

Yes, urlencoding would save the world, but the point in this is that after 
urlencoding, the file simply will not contain any chars that is outside ascii. 
Therefore, they will be stored into any filesystem without problems.

But the catch here is that the files in filesystem will not have "human 
readable" names, instead they have the urlencoded name. For example 
"test_file_ñ.png" would show up in filesystem "test_file_%C3%B1.png".

And this would mean in Mollify that you cannot enter filenames outside Mollify, 
and thus this solution has no value. I guess I could make this as an option for 
those who don't need updating/adding files outside mollify...

Original comment by samuli.j...@gmail.com on 8 Nov 2013 at 10:07

[GoogleCodeExporter]

This actually is interesting and i even think in my endless search for this 
problem, I have seen that article, but due to my limited knowledge i didn't 
know much what to do with it.

Although, yet again, i think your logic makes sense and this might not help in 
our case, something seems to be off just a bit here, just can't put my mouse on 
it.

Original comment by Logic...@gmail.com on 8 Nov 2013 at 10:08

[GoogleCodeExporter]

Yes, it makes sense now. The same goal can be achieved just by putting back 
UTF-8 in encoding settings, everything can be uploaded via Mollify, they just 
turn up like garbage on the filesystem, but that doesn't matter no more.

Original comment by Logic...@gmail.com on 8 Nov 2013 at 10:12

[GoogleCodeExporter]

Published version 2.2.9, which will at least fix the navigation issue. I'll 
form up some FAQ post on the charset conversion issue, which I think I will 
leave as it is for now (unless someone points out something that actually 
solves this entirely).

Original comment by samuli.j...@gmail.com on 11 Nov 2013 at 9:47

• Changed state: Fixed