Incoming file conversion not correct

[MrChristmas2000]

When importing my old sequence into Nutcracker 4.0 there seems to be some errors occurring. I have produced a video explaining in detail what I am seeing. That video is here: https://www.youtube.com/watch?v=TqkqywHQM9Q

[AzGilrock]

I don't have audio at work so I could be making some bad assumptions. I saw you use the file conversion page and was wondering what you thought that did. It would have no effect on the currently open sequence. Also keep in mind that the old way the program worked you could end up with data in the xseq file that was not represented in your XML file. That's because the old way you could somewhat imprint an effect into the data if you added an effect, rendered, and then removed the effect. So if you then take the resulting XML into 4.0 you won't end up with the same data you had before because that effect that was imprinted is no longer in the XML file. In that case the only way to get the same thing is make sure the old xseq data is included. I would try adding your old xseq file as a data layer underneath the nutcracker layer. There also could be some render order differences. In some cases maybe we are doing it wrong now and need to fix it but there will be cases where the way things rendered in the old version didn't make sense.

But like I said these are just thoughts I had without being able to hear what you were saying so I could be off base. Just some suggestions that may help you.

[MrChristmas2000]

It probably would be better to listen to the audio, maybe next time I will subtitle it for clarity.

The way ALL my sequences were created are as follows: 1st I have all my sequences created in LSP. 2nd I add additional effects on my RGB elements using Nutcracker, but keep some RGB effects created in LSP. 3rd I export those sequences to a .xseq file using LightElf, Supposedly Light Elf will export to a .fseq file but never got it to work doing that. 4th I would copy that exported file over to my Nutcracker working folder. 5th I would create a sequence file in Nutcracker 3.x adding in RGB effects as desired. At that point it was my understanding from Sean that when I did a save from the NC sequencer tab that NC would only overwrite the part of the .xseq file related to the newly added Nutcracker effects.

Now are you telling me that Nutcracker 4.0 will operate differently. If so that is going to be a MAJOR problem for me and probably a number of other people once 4.0 has been released.

[AzGilrock]

I was planning on listening when I can just wanted to offer some advice earlier than that. I don't think its going to be a big deal to have a workflow that works for you. The best way is to do what I suggested and create a data layer that points to that xseq file. Then all you gotta do is go to the data layer tab under Sequence Settings and hit the ReImport button whenever you make a change to the source data and then either hit Render All or Save to force the program to overlay the NC effects over that data. The other way is the .fseq file needs to exist before you open the sequence. Maybe I didn't see it correctly but it looked like you opened the sequence first and then did the convert. We've already opened the file at that point so we don't pull in data from the fseq again after it's been opened. I'd need to double check the source but I think you can also select the xseq file when you choose to open and it will pull in all that data and do the conversion on the xml file as long as the names match.

[AzGilrock]

By the way here's why the Data Layer method is better than the old way. If you just convert a file to fseq/xseq and use it as the underlying data and then lets say you have an effect that goes from 5 to 10 seconds. Then you decide to drag the end mark of the effect from 10 seconds to 7 seconds. Well if you rendered inbetween then you just destroyed 3 seconds worth of the underlying data. Then you need to re-create the underlying data and reopen the file to fix it. I ran into that all the time as a LOR user. I had to keep re-executing the lengthy conversion process. So I provided the data layer capability as a better method. Then you just point a data layer to the source file and when we generate the fseq file it reads the data for that layer in and then overlays effects so you never destroy the underlying data. And if you change the source file you just hit the ReImport button because we've saved the path info to that source file.

I wasn't involved with the development of the older version but I think when you hit save it was reading the fseq file, overlaying the NC effects, and then writing the fseq back out. We don't re-read the fseq file on Save. We've already got the data stored internally so that might be one fundamental difference.

[dkulp]

FYI: if the Nutcracker layer is the ONLY defined layer then it SHOULD work identically to v3. If you do define other layers, then it will clear everything.

[AzGilrock]

But I think the difference is you can't update the FSEQ file while the sequence is open and expect that data to be in the file when you save whereas that was a valid workflow in the old version. I'd have to look at old source to verify.

And that's why the layer is better...you WANT it to clear everything to fix any blemishes you create while you are dragging and deleting effects. It clears everything but you don't care because its all saved in the data layer that gets merged in when you save.

[dkulp]

Is there any way you could create a very short (2-3 seconds) sequence in LSP that manipulates those Star elements and such and do the convert/etc... steps and see if the same Preview problem exists? If so, send along the entire thing (xlights_*.xml, the x/fseq, etc...) I'm really at a loss on that one without even a thought about what could cause that.

The "directory" that comes up from the Convert tab is completely separate than either of the directories on the Setup page. It "remembers" the last directory that was used for conversions and will always pop up there. For those of us that always convert from HLS's export dir or lightElfs export location, this helps.

The crash backtrace is really strange. No idea on that either.

[MrChristmas2000]

Dan Ok, I'll work on the test sequence with just the star element. If you locate what is going wrong there it might also solve the other missing element effects.

Gil I am going to have to wrap my mind around the your recommended process. As I said I have two incoming files for conversion. A .xml and a companion .xseq file. Are you telling me to convert the incoming .xseq to a .fseq file. Convert the incoming .xml to a ver 4 .xml (done automatically when opened). Save the effects created by the new ver 4 .xml which will save as a .fseq data file. Now you mention above something about a Data Layer method. Can you elaborate a little on this. Does that mean that I need to create a model that covers all the other DMX incan channels in my sequence? For the life of me I can't find a way to import another file once the .xml sequence is opened. Maybe you need to do a step by step walk through.

[AzGilrock]

First I forgot to mention you do an awesome job creating videos. We need you for creating tutorials. :)

So I'm sure we probably do have some issues to work out so this suggestion for using the data layers wasn't meant to fix those but I think it will be the recommended way to use imported data. The way the program used to work caused Sean a lot of user support so that's why I'm steering people away from the old way. I really only works for people that don't do much in xLights after they bring in their data. If you already have a set of effects and don't need to drop down and edit new ones then the old way works fine. But if you are going to be changing effect lengths or deleting any effects using a data layer will be less headaches for you.

So for one this video I did a few weeks ago will show the process of importing a data layer: https://www.youtube.com/watch?v=vnOkKnz6z_A

You shouldn't need to convert your LSP data to an FSEQ file. I would leave them as XSEQ and use that as your data source for the layer import. So the process would be: 1) You have all your sequences created in LSP. 2) Export your sequences using LightElf to any working directory you want into XSEQ format. 3) Copy your old v3 XML to a new working directory. 4) Open xLights v4 and set your show directory to that directory. 5) Open the v3 XML and let it convert and close the dialog and check the sequencer tab to see your effects. 6) In the menu select Settings and click the Data Layers tab. 7) Click the Import button and navigate to where your LightElf export files reside. You don't even need to copy them to the xlight working folder if you don't want. 8) Select the XSEQ file you want to import and wait till the number of channels field quits saying its processing the data and shows you a channel count. 9) Now to see that data show up in Preview you need to either do a Save which will create the FSEQ file or you can do a Render All which should allow you to see it in Preview without doing the save yet. 10) Add your additional NC effects saving whenever desired. 11) If you LSP data changes just re-export using LightElf, then open up the sequence in xLights, got the Sequence Settings->Data Layers tab and select that data layer and hit the Re-Import button. It will pull your changes into the file again. Remember to hit Render All or Save to see the changes show up in Preview. The reason we don't automatically render is because it can be a lengthy process so we wanted to give you control over when that happens.

[MrChristmas2000]

Gil,

I am willing to work on some training vids, no problem. Thanks for the compliment, that was just a quick rough problem demo.

I got to step 9 but when I went to do a save Nutcracker crashed. There was not a zip crash file created but there I found and zipped what had been put in the temp folder I have zipped it and it is available here: https://dl.dropboxusercontent.com/u/28041968/Nutcracker%20Testing/xLights_dbgrpt-6752-20150317T202230.zip

Are the 'Change' buttons under the setup tab functional yet? I can't seem to get them to stay set to the folders I want? If this is a problem I will separate it out as another issue.

[AzGilrock]

Yeah I haven't looked into those folder buttons. I've mainly been using the far left button beside New Sequence that will let you change the Show folder and it updates both of those.

If you want to give me the xseq file that you're importing I could try it out. I think I have the other files I need from that crash report.

[MrChristmas2000]

It is here: https://dl.dropboxusercontent.com/u/28041968/Nutcracker%20Testing/Carol%20of%20the%20Bells%20-%20MASTER%20FILE_12_09_2014.xseq

How do you put a flag on this post to indicate that it is a XL 4.1 related?

[AzGilrock]

I think Sean has to add those kind of tags to the issues.

So I did find one bug from testing your sequence but not ones you found. What happened to me is since I didn't have the audio file with the same name I found a bug where it was trying to use the name found in the XSEQ file so I'll be checking in a fix for that in a minute. You definitely have something strange going on with some of those channels. I actually thought the stars looked incorrect in the old version also. If you had the same channels 305, 306, and 307 mapped to both objects I would expect them to light up in unison but they didn't in either version. Or am I missing something here. But there definitely were issues in some of the other areas you pointed out. We'll have to look into that.

[MrChristmas2000]

You are correct on the star channels. Each of the three individual models is mapped to 305, 306, and 307 and the combined custom model is mapped to the same three channels but they behave differently in the preview window. Once I get to a point there is no crash on save of this newly combined file I can see if this new merging of the file now responds the same as the old setup. I also now believe that the channel problem existed in 3.6 as well. I had noticed some behavior in my display this year that I had blamed on bad controllers but now believe that was actually problems in the sequences.

[dkulp]

I'm looking at the star thing.... It's a problem with the single channel star model. The Star model was really designed for RGB nodes and we've had a bunch of issues when NOT use RGB. He just found another.

[MrChristmas2000]

Ok, now I am feeling a little better. There is a number of elements that can be represented by either RGB elements or incans. Candy canes is another one in that category.

I tend to find the out of the ordinary bugs. LOL.

[AzGilrock]

I forgot to mention I was able to import your file as a data layer and then do a save. The Preview still had the issues. I'm curious what's happening also because I had several single color strings in my sequences last year and didn't run into any issues other than wrong colors showing in the Preview.

[MrChristmas2000]

I got the file to import but there is still a problem with what should be showing up in the 'Preview'. Elements are missing for one thing. The 'Star' seems to be now in sync with the custom model but it is not behaving as expected. It is effects to show up on a 3d incan star comprised of 3 star parts and the effects should be making the star appear to spin. The import shows only one of the stars elements flashing. I will put a side by side video so I can also point out the other elements that now do not show up with this version of Nutcracker.

[AzGilrock]

I don't think we did any changes that were going to fix your Preview issues yet. I'm not sure exactly what you were going to show side by side but what might be interesting is take just the xseq file without any NC effects added and show it playing in version 3.6.15 versus the new version. It should just be playing raw data in that case so we can see if there's something wrong with that.

[dkulp]

The "new" build makes sure the 2 star models are using the same channel number. In the old build, the "custom model" was properly displaying the data in channels 305-307. The 3 individual stars were displaying the data in channels 355-357 (the star model was improperly using the "# of lights per string" setting). That's why they were showing different things. If they aren't displaying the correct pattern now, then the channel assignments are wrong or the export from light elf is wrong.

[MrChristmas2000]

Ok, video is now completed and posted here:

https://www.youtube.com/watch?v=TnmAeSWcWoA

Files associated with this test is located here:

https://dl.dropboxusercontent.com/u/28041968/Nutcracker%20Testing/Carol%20of%20the%20Bells%20-%20MASTER%20FILE.zip

[dkulp]

More problems with "single channel white" nodes. :(

Hopefully fixed in the next build.

[MrChristmas2000]

Installed the first public Beta 4.0 release and it looks like the effects are responding as expected. Now I will test it out on all my other sequences. Thanks for both of your persistence in fixing this.

[AzGilrock]

I think this issue has been fixed but I'll leave it open for now and let you close it if you agree.

[AzGilrock]

I'm closing this tomorrow unless someone speaks up and says it's not fixed.

[MrChristmas2000]

Fine with me. I have not had any further importing problems. Thanks for the help.

[MrChristmas2000]

I have had no further importing problems.

Thanks for your help,

Tom Langley

From: Gil Jones [mailto:notifications@github.com] Sent: Thursday, April 09, 2015 10:46 PM To: smeighan/xLights Cc: MrChristmas2000 Subject: Re: [xLights] Incoming file conversion not correct (#80)

I'm closing this tomorrow unless someone speaks up and says it's not fixed.

— Reply to this email directly or view it on GitHub https://github.com/smeighan/xLights/issues/80#issuecomment-91409508 . https://github.com/notifications/beacon/AC7giweQ5Vd4u3tA7LO7bi8WS9waEVqzks5n9zD-gaJpZM4DwII8.gif

[AzGilrock]

Ok just wanted to make sure there wasn't something we didn't realize still not working. Will close it now.