search

fdbozzo / foxbin2prg

Visual FoxPro 9 Binary to Text and Text to Binary converter. Replacement for SCCText(X) and TwoFox that is bi-directional (Text is editable)
https://vfpx.codeplex.com/wikipage?title=FoxBin2Prg
MIT License
59 stars 35 forks source link

Trailing spaces are lost for memo fields converting dbf with including data. #81

Closed bjornhoeksel closed 2 years ago

bjornhoeksel commented 2 years ago

ℹ Computer information

📝 Provide detailed reproduction steps (if any)

Trailing spaces are lost for memo fields converting dbf with including data. Convert SpaceBug.DBF to reproduce problem I fixed this problem in added foxbin2prg by replacing space with a placeholder and restore them later.

✔️ Expected result

Keep Trailing spaces for memo fields

❌ Actual result

Trailing spaces are lost for memo fields

📷 Screenshots

foxbin2prg.PRG.txt SpaceBug.DBF.txt SpaceBug.FPT.txt

lscheffler commented 2 years ago

@bjornhoeksel There are two problems. .ENCODE cleans up trailing spaces, and the TEXT TO construct (only this one) removes trailing spaces if the spaces are the only signs on a last line.

So I look up TEXT TO.

Aunt Edit says: replaced ENCODE with TEXT TO

lscheffler commented 2 years ago

ok. Now I'm confused

The only working part on your code is the removed RTRIM. There is a problem in the TEXT TO, but it is not valid for memos.
.Encode(luValue, .T.) wraps luValue in <![CDATA /> tag anyway, so the trailing space is preserved.

I just get the right db2 with the spaces by removing the RTRIM. The STRTRAN does nothing.

        <RECORD>
            <QRY_JOIN1><![CDATA[action.src_code = contact.src_code AND
action.act_who = qry_who.sc_code AND
action.act_type = qry_type.sc_code                                                                                                                                                                                               ]]></QRY_JOIN1>
        </RECORD>

Would you mind to check this?

lscheffler commented 2 years ago

The problem with TEXT TO is a problem for VARCHAR, if they are have trailing spaces. Char has spaces anyway, but on varchar one might store some spaces, and they will come out of the field, since this is a length, not a space problem.

Aunt Edit

bjornhoeksel commented 2 years ago

Hi Maybe some explanations with some some small samples makes tings clearer. Fix is two part remove rtrim to keep last spaces last line. And replace spaces with placeholder to restore them later. Is needed for memos because trailing spaces can matter.

Problem is with TEXT TO trailing space before line return are removed See sample prg’s correct and wrong. These places text on clipboard that you can paste. You will see by wrong that spaces are missing. correct.PRG.txt wrong.PRG.txt

lscheffler commented 2 years ago

@Bjornhoecksel Please send me the config used on the table. Please use DO FOXBIN2PRG.PRG WITH '-C','path-to-the-table\SpaceBug.cfg'
to create a config file with all settings.

If you use SpaceBug.DBF.cfg, this too.

bjornhoeksel commented 2 years ago

@lscheffler Hi

This is the config I used. Only change in this is that dbf files include data records

FOXBIN2PRG.CFG.txt

lscheffler commented 2 years ago

@bjornhoeksel
Please create the config as mentioned above. FoxBin2Prg has a chain of inheritance. I like to be sure to have the settings valid.

bjornhoeksel commented 2 years ago

@lscheffler SpaceBug.cfg.txt

Created fconfig file with command DO FOXBIN2PRG.PRG WITH '-C','c:\av win\SpaceBug.cfg'

c:\av win\ is location table

lscheffler commented 2 years ago

@bjornhoeksel
Thank you.
FYI. If you check the file you send me, there is a list of the inheritance. So, with the first file, I get my default settings involved. The last file you send sets ALL options, so I get your settings.

lscheffler commented 2 years ago

@bjornhoeksel

Hi Maybe some explanations with some some small samples makes tings clearer. Fix is two part remove rtrim to keep last spaces last line. And replace spaces with placeholder to restore them later. Is needed for memos because trailing spaces can matter.

Problem is with TEXT TO trailing space before line return are removed See sample prg’s correct and wrong. These places text on clipboard that you can paste. You will see by wrong that spaces are missing. correct.PRG.txt wrong.PRG.txt

Oh. I learned the problem in TEXT .. ENDTEXT. Only the problem is no problem here.

In line 26694 there is luValue = .Encode(luValue, .T.). The important part is the .T. . This wraps luValue into a CDATA tag (see around line 26736), that means the spaces are not trailing anymore. So the problem in TEXT .. ENDTEXT does not occur. So the remove of RTRIM keeps the spaces first, and .ENCODE wraps it. The problem occurs for VARCHAR and VARBINARY if they have trailing spaces stored. Then the data is not wrapped and cut off. For a normal CHAR it does not matter if the space is stored.

My solution is not to add (doubtfull) replace strings but to alter the TEXT .. ENDTEXT like

TEXT TO lcText TEXTMERGE NOSHOW flags 1+2 PRETEXT 1+2 additive
<<>>            <<'<' + lcField + '>' + luValue + '</' + lcField + '>'>>
ENDTEXT

this removes the cutoff of the trailing spaces by <<luValue>>, since luValue is now inside a concatenation.
The RTRIM is removed.

Please try the prg attached

foxbin2prg.zip

bjornhoeksel commented 2 years ago

@lscheffler Hi

I tried the prg file you send with spacebug.dbf But the result is incorrect for me. If I convert back to dbf with the new prg file you send The first 2 lines in memo have no space after AND

Result new prg action.src_code = contact.src_code AND action.act_who = qry_who.sc_code AND action.act_type = qry_type.sc_code

Result my version action.src_code = contact.src_code AND action.act_who = qry_who.sc_code AND action.act_type = qry_type.sc_code

This is a problem because this result in a expression that is not valid anymore in memo field because of the missing spaces.

bjornhoeksel commented 2 years ago

@lscheffler

The sample are not working in chat here all trailing spaces are eaten. But difference is that my version has space after and for first 2 lines

lscheffler commented 2 years ago

Maybe you generate some files with the data, zip and post (zip files work here)

bjornhoeksel commented 2 years ago

@lscheffler Strange I check the data of prg file and saw your change. I compiled it. I have no foxbin2prg.exe that is running instead. I copied new spacebug.dbf and check spaces are there. Run prg file and removethe .dbf file. run prg file with .db2. Result missing trailing spaces after and for first 2 lines. Last line has the spaces.

Maybe related to some vfp setting? But no idea which one.

bjornhoeksel commented 2 years ago

SpaceBug.zip @lscheffler Here the zip file with the table maybe some data is lost with send as .txt file

lscheffler commented 2 years ago

I have run DO FOXBIN2PRG.PRG WITH 'BIN2PRG','SpaceBug.DBF' against the table using the zip and the config you send. See attached, see line 47 with the spaces
This is the prg a send you. just a bit different TEXT ENDTEXT and RTRIM removed. SpaceBug.zip

lscheffler commented 2 years ago

@bjornhoeksel I have recreated the dbf with DO FOXBIN2PRG.PRG WITH 'PRG2BIN','SpaceBug.DB2', the space is in the memo.

lscheffler commented 2 years ago

I have run a test against varchar with trailing spaces, (creating text, deleting table, recreating table) and the right count of spaces is recreated

bjornhoeksel commented 2 years ago

@lscheffler Hi I downloaded the Spacebug .zp you send. But I see there is the same problem in de spacebug.db2.

I think we had some misunderstanding. The problem is not with line 47 but 45 and 46 a space is missing after the AND on those lines

Can you check that? The version I created keep these spaces but maybe not the prettiest solution.

lscheffler commented 2 years ago

Sorry, I closed by accident.

lscheffler commented 2 years ago

@bjornhoeksel
Those spaces. okay. I never recognized them at all. I look

lscheffler commented 2 years ago

Then I'm pro removing the whole TEXT .. ENDTEXT for char type and the likes and just concatenate the string in full lcText = lcText + 0H0D0A + ' <' + lcField + '>' + luValue + '</' + lcField + '>' This removes the oddness of TEXT - and as far as I see the flags it's just the same + with the trailing spaces

Looks like TEXT .. ENDTEXT is not meant for such operations.

See file attached SpaceBug.zip

I have expanded the table a bit, you can see it working with other types, in special keeping the two spaces in varchar

bjornhoeksel commented 2 years ago

@lscheffler Hi

I checked it and this change work good. Yes I think its best to avoid TEXT .. ENDTEXT in this situations to avoid strange behavior.

lscheffler commented 2 years ago

@bjornhoeksel new version is on Thor

bjornhoeksel commented 2 years ago

@lscheffler

Thanks!

bjornhoeksel commented 2 years ago

@lscheffler Hi I test the latest version. I saw still a problem. Now the spaces of last line are lost because of the rtrim. The rtrim was not removed in code for memo. The first 2 lines are ok now.

Code with problem: Fix is remove rtrim last line of this code snipped

! Changed by: SF 07.4.2022 ! ! Changed by: SF
! https://github.com/fdbozzo/foxbin2prg/issues/81 / Trailing spaces are lost for memo fields converting dbf with including data. ! RTRIM removed trsiling spaces from memo ! ! luValue = .Encode(Rtrim(luValue), .T.)

lscheffler commented 2 years ago

@bjornhoeksel How ever I managed to activate this line again.

lscheffler commented 2 years ago

@bjornhoeksel ReRun ...

bjornhoeksel commented 2 years ago

@lscheffler Yes! it works now correct