Closed GoogleCodeExporter closed 9 years ago
Definitely windows problem. I'm gonna dig out the windows virtual machine and
take a look at this (and the charset)
Original comment by samuli.j...@gmail.com
on 3 Nov 2013 at 7:57
I've installed Windows 7 virtual machine with XAMP server, and made some
changes. At least in my environment I am now able to navigate on different
levels and handle files (including upload) with chars like "ñäöå"
I've uploaded this version here:
https://code.google.com/p/mollify/downloads/detail?name=mollify_2.2.9b1.zip
It is marked as beta because the changes I made were quite big, and I need to
test it more before I release it. But if you could test it, it would be great.
I was using configuration setting
"convert_filenames" => "CP1252"
Original comment by samuli.j...@gmail.com
on 6 Nov 2013 at 10:28
This is pretty much incredible what you have done, but i still have few issues.
Namely i have a file called: (testing with that file seems to bring out all
sort of interesting issues everywhere) "Enya - La Soñadora.mp3". When i upload
that file (thanks to your update, i now CAN upload that file) it changes the
special letters to, as follows: "Enya - La Sońadora.mp3", notice the "n"
letter has changed. It uploads correctly and it can be viewed after upload and
even be used (downloaded etc) also the name viewed FROM Mollify displays the
name correctly "Enya - La Soñadora.mp3", BUT on the file system, viewed from
regular windows explorer, is on the disk with the name: "Enya - La
Sońadora.mp3".
This now results another issue that if i put the file "Enya - La Soñadora.mp3"
directly on the Mollifys user path on the local disk with simple copy/paste
action, then Mollify cannot read it anymore and results a "Request Failed".
Also it adds to PHP Error Log (debug disabled, tell me if you need the whole
shebang):
MOLLIFY ERROR: PHP error #2, is_dir(): open_basedir restriction in effect.
File(V:\Local_Directory_on_Disk2\Mollify\blabla\Test\Enya - La So?adora.Mp3) is
not within the allowed path(s)...
I'm guessing the "?" breaks the configuration readability for PHP itself.
I haven't tested by changing the Apache charset to anything different than
UTF-8 because i would still need to have UTF-8 enabled in Apache. Also PHP is
set to have UTF-8 enabled for my other scripts that need/might need it, not
mentioning MySQL, they all have UTF-8.
Before i tested i DID put the:
"convert_filenames" => "CP1252",
and i had "charset on:
"charset" => "utf8",
So basically reading the files from disk which haven't been put there VIA
Mollify, is still an issue, at least for me and that is the major problem here.
One more positive note is that the folder navigation now works correctly.
Please let me know if you need more information.
Original comment by Logic...@gmail.com
on 7 Nov 2013 at 6:56
It is clear that the conversion is not correct yet. Whenever some char ends up
different, there is wrong conversion done. And of course it is natural that
when even one char is converted incorrectly, the file is not found. It's the
same as replacing all "a"'s into "b", the files simply don't exist with those
names.
The question is, what is the correct conversion. Are you sure it is supposed to
be "CP1252"? I'm no expert on a) windows or b) charsets in general, and cannot
identify what is the charset that has "ń" where CP1252 has "ñ".
Basically you would have to create a test php page that lists the files in this
particular folder, and makes a conversion. There you could try out different
encodings to see which one produces the correct utf-8 result. I don't have this
kind of test page, don't know if I had time to make one...
The most important thing here is that, _everything_ must operate on utf-8. PHP,
Apache, MySQL and website. Mollify assumes that all strings are utf-8. And only
in case of windows, the last layer must be converted, which is the filesystem.
I think my own server works as a proof of concept that conversion can be done,
since for example that exact filename "Enya - La Soñadora.mp3" works, both
when entered directly into windows filesystem, or entered via Mollify app.
btw, your error is again the "open_basedir", same as with the hashing problem.
You removed the beginning of the path (for privacy reasons, of course), did you
check that the rest of the path was actually correct (excluding the ? mark in
filename)?
Original comment by samuli.j...@gmail.com
on 7 Nov 2013 at 12:46
OK, so I made the test script, didn't seem too complex. Just copy it into your
web server, edit it and modify the variables. Try the folder where you have the
problematic file, and see what the browser says.
Now this script does make the same conversion as mollify, so you should see the
file names exactly as they are in filesystem. In my windows server, CP1252 does
output the letters correctly, including "ñ".
Original comment by samuli.j...@gmail.com
on 7 Nov 2013 at 1:05
Attachments:
This is driving me insane.
Can you send me your configuration files on PHP, Apache and MySQL, also the
version numbers and are they for 32bit or 64bit architecture? Also what version
of Windows are you running this on and what are the regional and language
settings?
I tried a lot of charsets and the closest i got was "CP1257" which did display
some symbols what "CP1252" didn't show, but STILL, can't encode correctly the
"ñ".
Since it's working for you and not for me, I'm guessing there has to be
something wrong with my configuration somewhere in Apache, PHP, MySQL or
Windows in general.
Original comment by Logic...@gmail.com
on 7 Nov 2013 at 6:55
Did some more testing and it seems it also can't translate random russian
letters, for example "фртыйкмвхжэщъыяёсёё" and replaces them
with "?" marks. I'm assuming it can't do half the stuff in the world and i
can't figure out what conversion must i use. Also it doesn't make sense your
conversion charset works, because in that charset "CP1252" there are no russian
letters...
Something else in here is fishy
Original comment by Logic...@gmail.com
on 7 Nov 2013 at 7:11
Not sure if this helps, but i changed the settings in your script from "CP..."
to UTF-8 just randomly to test some things and then i got an error log not from
Mollify, but from PHP itself with:
PHP Notice: iconv(): Detected an illegal character in input string in
C:\PATH_TO_MOLLIFY\charset.php on line 16
I'm not sure what this is suppose to mean, as i didn't change the line 16, but
the line 4.
Original comment by Logic...@gmail.com
on 7 Nov 2013 at 7:18
I installed WAMP server myself on a Virtual Machine and i STILL have the same
problems in there, i tested only with your charset script.
I didn't modify any other settings and used WAMP servers default settings. How
the hell is that even possible now. I'm going insane.
It simply works for you and not for me.
Original comment by Logic...@gmail.com
on 7 Nov 2013 at 8:12
I got my friend to help test this out as well, i just got a random picture and
named it to:
"Sońa_Soña_Süõäö_€tere_фртыйкмвхжэщъыяёсёё_.jpg"
just as a test input string to see how it works. He didn't get it to work under
Windows server as well, under windows server it shows:
"Soña_So?a_Süõäö_€tere_???????????????????_.jpg" with the settings
CP1252. He got it working under linux environment while setting the encoding to
UTF-8, but it's clear anyway that it works under Linux. Also i set up, as i
said before, a VM with default installation of WAMP and didn't get it to work
either, the results were the same. So i am very curious what server
configuration you have that it works in your environment.
Original comment by Logic...@gmail.com
on 7 Nov 2013 at 11:25
Your example with russian characters did not work either on my server, the last
part is all question marks. So my charset is not CP1252 either, and from
reading posts it seems it should be UTF-16 in NTFS filesystem. But this did not
work either, not even close. So I can say that this really is unnecessary
difficult with microsoft products, both windows and IE cause so many extra
troubles, I wish I could just ignore them...
For example this page http://evertpot.com/filesystem-encoding-and-php/ has a
conclusion "It would be possible to provide a consistent experience, if it
weren't for windows. Windows does have all the proper api's to deal with
international filenames, but I can only assume PHP simply does not support
them. I do believe this was scheduled for PHP6, but now that's off the hook. I
hope the filesystem api's are replaced even before the entire language is
unicode-based."
Original comment by samuli.j...@gmail.com
on 8 Nov 2013 at 6:03
I too found the UTF-16 and it also did not work, i even tried loads of other
ridiculous solutions and of course, no help.
My friend also told me that this might be literally PHP issue itself.
I was waiting for your response and now I'm sad.
I set back the charset encoding to UTF-8 so at least people can upload anything
to the server, but i can't implement half my ideas i wish i could.
I guess we just have to wait for PHP6 release.
Either way, thank you so much for putting your time and effort in it, it was
hell, but at least now we know the answer.
Original comment by Logic...@gmail.com
on 8 Nov 2013 at 6:27
One last trial I'm gonna make. According to this post
http://stackoverflow.com/questions/1525830/how-do-i-use-filesystem-functions-in-
php-using-utf-8-strings it should be possible to just urlencode the utf-8
filenames, which the filesystem should accept and interpret according to own
charset.
I have a doubt that if you just urlencode utf-8 char, it will simply transform
any non-ascii letter into its byte values, for example "ñ" -> "%C3%B1". These
values are utf-8 byte values in hex, and I'm really wondering that how can this
work regardless of target charset? I mean is the C3+B1 always "ñ" in any
windows filesystem no matter if it is in chinese locale etc?
But we'll see, I let you know how it turns out.
Original comment by samuli.j...@gmail.com
on 8 Nov 2013 at 10:02
Oh no, just realizes after pressing submit, that this is for entirely different
goal.
Yes, urlencoding would save the world, but the point in this is that after
urlencoding, the file simply will not contain any chars that is outside ascii.
Therefore, they will be stored into any filesystem without problems.
But the catch here is that the files in filesystem will not have "human
readable" names, instead they have the urlencoded name. For example
"test_file_ñ.png" would show up in filesystem "test_file_%C3%B1.png".
And this would mean in Mollify that you cannot enter filenames outside Mollify,
and thus this solution has no value. I guess I could make this as an option for
those who don't need updating/adding files outside mollify...
Original comment by samuli.j...@gmail.com
on 8 Nov 2013 at 10:07
This actually is interesting and i even think in my endless search for this
problem, I have seen that article, but due to my limited knowledge i didn't
know much what to do with it.
Although, yet again, i think your logic makes sense and this might not help in
our case, something seems to be off just a bit here, just can't put my mouse on
it.
Original comment by Logic...@gmail.com
on 8 Nov 2013 at 10:08
Yes, it makes sense now. The same goal can be achieved just by putting back
UTF-8 in encoding settings, everything can be uploaded via Mollify, they just
turn up like garbage on the filesystem, but that doesn't matter no more.
Original comment by Logic...@gmail.com
on 8 Nov 2013 at 10:12
Published version 2.2.9, which will at least fix the navigation issue. I'll
form up some FAQ post on the charset conversion issue, which I think I will
leave as it is for now (unless someone points out something that actually
solves this entirely).
Original comment by samuli.j...@gmail.com
on 11 Nov 2013 at 9:47
Original issue reported on code.google.com by
Logic...@gmail.com
on 2 Nov 2013 at 11:23Attachments: