chaddro / EasyPopulate-4.0

Data import and export module for Zencart 1.3.x and 1.5.x
GNU General Public License v2.0
24 stars 31 forks source link

Categories delimeter #26

Closed mesnitu closed 8 years ago

mesnitu commented 8 years ago

$category_delimiter = "\x5e" , doesn't seen to be splitting and ends up in names length error Changing to : $category_delimiter = '\x5e' , it's ok Categories example: select the forum that you want^to visit from the selection

mesnitu commented 8 years ago

Please ignore this last issue. It's a issue for me, but I'm getting all kind of different results, so what I've mentioned above, it's not ok either.

mesnitu commented 8 years ago

Ok, it's a issue out of the box. Started from scratch : Deleted all admin files and downloaded them again from the shop with Easy Populate 4.0.22 - Beta 6-10-2012 ( just the EP4, no bookx fields )

Export all products, import all products -> OK

Deleted all related Easy Populate 4.0.22 - Beta 6-10-2012, and installed the Easy Populate 4.0.33a - Beta 02-29-2016 ( no bookx )

Import the same file -> Not Ok

SKIPPED! - Model: 9789898087973 - Category name: "Desenvolvimento Pessoal^Espiritualidade" exceeds max. length: 32
SKIPPED! - Model: 9789897070372 - Category name: "Infantil/Juvenil^Infantil / Juvenil" exceeds max. length: 32

Not only this skipped cat because of name length, but actually is not splitting any categories at all. All the others that have child categories are not split , but they don't exceed the length

Installed Languages
2-pt: Portugues
Default Language: 2-Portugues
Internal Character Encoding: UTF-8
DB Collation: utf8
mesnitu commented 8 years ago

If I use what' s on Easy Populate 4.0.22 import, the cat gets split : $categories_names_array[$lang['id']] = mb_split('\x5e',$items[$filelayout['v_categories_name_'.$lang['id']]]);

On this version, only using explode $categories_names_array[$lang['id']] = explode($categories_delimiter,$items[$filelayout['v_categories_name_'.$lang['id']]]);

Opening on libreoffice, the character is ut8 , opening on notepad++ also says utf8.

mc12345678 commented 8 years ago

On your EP4 admin screen upper right hand corner, do you have the following message:

'mb_internal_encoding not available'

Is your store using UTF8? Database, language, etc..?

mesnitu commented 8 years ago

I don't see that , just this ( plus the database and installed language that I posted above :

Configuration Settings
Upload Directory: admin/temp/
Verbose Feedback: TRUE
Split Records: 2000
Execution Time: 300
Convert Curly Quotes: No Change
Convert Char 0x92: Basic
Enable Products Metatags: 1
Enable Products Music: 0

If I check this :

echo mb_regex_encoding();

            $categories_names_array[$lang['id']] = mb_split( $categories_delimiter, $items[$filelayout['v_categories_name_' . $lang['id']]]);

I'll see this :

UTF-8
Array
(
    [2] => Array
        (
            [0] => Desenvolvimento Pessoal^Espiritualidade
        )

)
mesnitu commented 8 years ago

All database is in utf8_general_ci, except some tables like Ceon , and reward points, that have latin_1

mc12345678 commented 8 years ago

If I use what' s on Easy Populate 4.0.22 import, the cat gets split : $categories_names_array[$lang['id']] = mb_split('\x5e',$items[$filelayout['v_categoriesname'.$lang['id']]]);

On this version, only using explode $categories_names_array[$lang['id']] = explode($categories_delimiter,$items[$filelayout['v_categoriesname'.$lang['id']]]);

Opening on libreoffice, the character is ut8 , opening on notepad++ also says utf8.

So saying that both of the above work without issue, but not a mix of the two?

mesnitu commented 8 years ago

On mb_split , using $categories_delimeter.... doesn't . _THis works _

$categories_names_array[$lang['id']] = mb_split('\x5e',$items[$filelayout['v_categories_name_'.$lang['id']]]);

$categories_names_array[$lang['id']] = explode($categories_delimiter,$items[$filelayout['v_categories_name_'.$lang['id']]]);

_This doesn't _ $categories_names_array[$lang['id']] = mb_split($categories_delimiter,$items[$filelayout['v_categories_name_'.$lang['id']]]);

mesnitu commented 8 years ago

But it looks like I'm the one with this issue....

mesnitu commented 8 years ago

Just one more thing: at exporting this categories, they get split

mc12345678 commented 8 years ago

To clarify, if while using the current software you export a file then turn around and import the file, then there is no problem? If you export a file then perhaps modify/create a category combination, is it successful or does the problem return?

mesnitu commented 8 years ago

The only way it works is using the explode or mb_split('\x5e'

What I didn't test while testing with EP022 was to replace the '\x5e' with $categorie_delimeter to see what happened.

But this is some sort of encoding issue, very hard to detect ,I imagine. I'm on windows 10 ( while kde5 matures a bit ), using LibreOffice. Database is utf8, opening the files is utf8.

But even the opening file doesn't matter, cause if I do a export / import directly, it doesn't work either. So could it be on writing the file from database to the the folder ?

But I had to move on. In the bookx genres, I'm actually using the '\x5e', so the genres, authors, etc, are split.

So... don't know what to say more. I've done some mb_detect_encoding,etc... all return utf8

But if no one is issuing this, here or the forum....don't see how to replicate this.

`

mc12345678 commented 8 years ago

This issue has been corrected through the use of preg_quote on $categories_delimiter in the import code in version 4.0.34.a specifically at commit: 23d9b848cfeee30c300b2e940948651b30a62005.