write db to files function?

[jools-r]

I know this is in development so I'm probably jumping the gun – also not sure if "issues" is the right place to post this but anyway here goes:

How about a "write db rows to files" function similar to cnk_versioning? I can see you have a write to file function in the source but as far as I can tell it's not (yet) referenced anywhere. I realize that if you have your own stripped-down custom txp-setup with only the mandatory forms that this isn't so essential, but sometimes it is useful as an "export" possibility.

On a related note, deleting a file doesn't currently result in the relevant entry in the db-table being deleted. To a certain extent, that's good: it means you can disable the plugin and hand over a correct installation to the client. I guess this would also be dependent on writing all db-rows to files as well or else they'll all be deleted. On the other hand, there's no way of housekeeping, for example if you decide to rename a form or want to get rid of old cruft, duplicate or alternate forms etc. I guess one needs to disable the plugin, do housekeeping the respective tab, then switch it back on...

[gocom]

Issues is a fine place for these - especially for now when there is no other place to post to ;-)

Deleting (and overwriting items with empty content) is disabled by default. It should be configurable with the <delete/> and <ignore_empty/> properties as in the example config. Or should if those work.

As you note, both are disabled for usability and security reasons. I've used cnk_versioning in the past and one thing that plugin did me more than once was trashing the table. Mainly because it doesn't check that directories exists (or are readable), but also because importing can happen before exporting (where did my content go?).

You say export function? Are you mind reader, or do you mean the rah_flat_files::write()? ;) That's there because it's general filesystem class, and I happened to write it there.

There used to be export function in the source (you mind reader ;p) when it first was developed internally, but the problem that was faced was the different formats. As you may have noticed, the plugin supports different file formats, but it's unclear to the plugin which DB field goes where and what specific file should contain. For instance, some fields can not be exported, but should be let to MySQL/Textpattern to handle.

One thing is also validity. XML fields can only take a string. All content should be checked for invalid content and markup fields wrapped in CDATA blocks. But what happens when that field in DB contains CDATA tag? Leads to invalid XML and impossible XML parsing.

Dunno about exporting. It's potentially never going to be error free. It's like checksums. Mostly you are fine, but there is the change that different content gives out same hash.

I do agree, exporting is something that I would want too (as a runnable task). Experimenting with the best method. It likely will end up (if it does end up) in its simplest form; exports all the fields from the table (in xml/flat_meta format), building somewhat valid XML and potentially causing malformed, unusable files.

[jools-r]

All good comments.

I must admit I didn't see <delete/> and <ignore_empty/> so I've not tried them. Yes, I meant rah_flat_files::write() but couldn't see where it was being referenced. I half guessed it was in preparation for something that might come later.

I'm not familiar with intricacies of the xml file structure, but did wonder whether badly-formed code within an xml field could bust the whole thing (that's what the CDATA is for I guess so you'd have to remove the outermost CDATA wrapper each time on import). That was partly the reason why I tried flat file handling of just the code window of smd_macro (the "definition" column). It works as long as you don't edit code in the window and as long as you reload any changes you made to the flat file before pressing "save" in the smd_macro pane. However, sooner or later I'm bound to forget and overwrite my code/save the wrong code so I've gone back to working in the browser for the moment.

Otherwise rah_flat's configurability makes it potentially very extensible for handling data from other systems with txp. I had another project a short while ago that could have been fed with xml files exported from a software system (an estate agent software). rah_flat might have been a good solution, particularly if it tracked deleted files and removed the corresponding db entries. In the end we did find a way to directly get that data into a separate mysql db table.

So, if you do find a good way of doing an export, that would be excellent!

Am 20.02.2012 um 15:08 schrieb Jukka Svahn:

Issues is a fine place for these - especially for now when there is no other place to post to ;-)

Deleting (and overwriting items with empty content) is disabled by default. It should be configurable with the <delete/> and <ignore_empty/> properties as in the example config. Or should if those work.

As you note, both are disabled for usability and security reasons. I've used cnk_versioning in the past and one thing that plugin did me more than once was trashing the table. Mainly because it doesn't check that directories exists (or are readable), but also because importing can happen before exporting (where did my content go?).

You say export function? Are you mind reader, or do you mean the rah_flat_files::write()? ;) That's there because it's general filesystem class, and I happened to write it there.

There used to be export function in the source (you mind reader ;p) when it first was developed internally, but the problem that was faced was the different formats. As you may have noticed, the plugin supports different file formats, but it's unclear to the plugin which DB field goes where and what specific file should contain. For instance, some fields can not be exported, but should be let to MySQL/Textpattern to handle.

One thing is also validity. XML fields can only take a string. All content should be checked for invalid content and markup fields wrapped in CDATA blocks. But what happens when that field in DB contains CDATA tag? Leads to invalid XML and impossible XML parsing.

Dunno about exporting. It's potentially never going to be error free. It's like checksums. Mostly you are fine, but there is the change that different content gives out same hash.

I do agree, exporting is something that I would want too (as a runnable task). Experimenting with the best method. It likely will end up (if it does end up) in its simplest form; exports all the fields from the table (in xml/flat_meta format), building somewhat valid XML and potentially causing malformed, unusable files.

---

Reply to this email directly or view it on GitHub: https://github.com/gocom/rah_flat/issues/2#issuecomment-4054101

[gocom]

that's what the CDATA is for I guess so you'd have to remove the outermost CDATA wrapper each time on import

Rah_flat does use XML parser to parse XML files. It fully understands XML structure, but also requires that the XML used in the flat files is valid. For example all HTML special character will need to be escaped or the field (node) wrapped in CDATA block.

[gocom]

Closing as this is about the old code base.