Add support for single-file sketches

[lmihalkovic]

At the moment a sketch is a folder with a file of the same name with .pde /.ino as extension. Might be nice to add .inz/.pdz file support where the file would be a simple ZIP archive of the sketch. With the editor code cleaned up, there is no reason why the IDE would not be able to deal with directly as archives as opposed to opening them. The distro would then be smaller on disk with the the examples shipped as .inz archives. When save-as, the modifiable copie of examples would then be expanded as things are now.

The feature could then be used to help migrate seamlessly between Create and IDE. Not to mention users able to email INZs around.

[ffissore]

AnalogReadSerial examples takes ~61K of disk space. Its zipped version is 57K. The amount of work needed to support such an optimization is tremendous. Hence such feature is useless.

As I've already told you, I love your enthusiasm. I'm not sure I told you that I think you're using it the wrong way and you may end up exhausting it. Please give a read to this http://www.joelonsoftware.com/items/2006/12/09.html

[lmihalkovic]

I think we already reached the conclusion that, like many things, "tremendous" is in the eyes of the beholder. E.g adding support for images took 1/2 day because i first had to clean the internals of Editor/SketchCodeDoc/SketchCode. But now that it is done, markdown came almost for free (except for completion). And the reason why it took 'only' 1/2 day is the very first cleanup i did: removing all the if(xxx.endsWith("ino")|"pde"|...) sprinkled all around. You do that, and replace it with a somewhat decent design (I am quite sure many will find obviously improvable things about what it did when the code goes to my branch), then things that look like roadblocks start to lineup, and making bigger changes is now easy.

Joelonsxxx is full of interesting ponderings... I will not attempt to turn the argument around and compare features for features with Create.. the company has clearly decided that IDE is to be but a stepping-stone into the bigger, grander new product. I fully understand the strategy, even if as a user I don't like it.

I would like to HUMBLY (no sarcasm) offer the following observation from my past experiences. Every code base starts with a simple, sometimes even elegant, design with lots of consistance. As time passes, more people bring their input, the cohesiveness dilutes (nobody keeps the complete mental picture anymore), the elegant design becomes a family of designs, until eventually the anything-goes stage is reached. At that point, any new feature, as small as it might be, carries a high implementation cost and a heavy risk of regression. Teams gradually settle in the thinking that a new app is needed (i am not refering to the legitimate company decisions to move into next directions). Somewhere before the point of discouragement and apathy is a magic spot where the code is not complete trash-can material yet, and where with a bit of back paddling, a lot of un-incumberred forward motion is again possible.. This is the place, when a team/poduct can reach it, where the developers can have the highest productivity, and knock out seamingly big things quite easily (again SIMPLE is rarely synonymous for OBVIOUS).

I am not trying to convince anyone of anything, and do not pretend to hold any wisdom for the ages

[lmihalkovic]

here is a small concrete example

    // TODO "cut" and "copy" should really only be enabled
    // if some text is currently selected
    JMenuItem cutItem = newJMenuItem(tr("Cut"), 'X');
    cutItem.addActionListener(new ActionListener() {
      public void actionPerformed(ActionEvent e) {
        handleCut();
      }
    });

With the current codebase, this simple TODO has as much chance of ever getting addressed as my patches have of ever making it into master.

This is my version

    menu.add(new EditorCodePasteboardAction.EditorCodeCutAction());

which relies on

public abstract class EditorCodePasteboardAction extends AbstractAction implements EditorAction, ArduinoDynamicAction {
}

The change is small, and you might even say 'why bother' The consequence of the change (and more exactly like this one) are

• Editor.java is free of textarea/scrollpane (which is why it was so easy to add an image editor, and the reason why I can now look into support LUA for the main sketch file)
• Any action that implements the ArduinoDynamicAction is automatically refreshed before being displayed
• Actions can have a different keybinding (I won't do it)
• I am fixing the 'magic' listener in SketchCodeDoc
• I can easily find all the actions at runtime or in the eclipse

blah blah blah... the point is that IMHO the code is a lot cleaner (one file one purpose), simpler to read, and trivial to extend... more with less

  public static class EditorCodeCopyAction extends EditorCodePasteboardAction {
    public EditorCodeCopyAction() {
      putValue(NAME, tr("Copy"));
      putValue(ACCELERATOR_KEY, KeyStroke.getKeyStroke('C', SHORTCUT_KEY_MASK));
    }

    @Override
    public void execute(ActionEvent e, Editor editor, SketchTextArea textarea) throws Exception {
      textarea.copy();      
    }    

[q2dg]

I just want to say that the idea of matching the offline IDE's functionality with online IDE's one is a real future investment that should be taken into account and not be thrown away with a simple "Will not fix". Especially if there is already implemented code ready to merge...so let's discuss/prove every change individually and give it a chance...it is not so hard. There are many people who (will) prefer the offline IDE and if it does not follow the same steps online IDE do, they will be treated (or will be felt) as second-class citizens. From here my support @lmihalkovic work.

[cmaglie]

Especially if there is already implemented code ready to merge

@q2dg Where is this "code ready to merge" exactly?

There are many people who (will) prefer the offline IDE and if it does not follow the same steps online IDE do they will be treated (or will be felt) as second-class citizens

Some feature may require a lot of work to be implemented on the Java IDE, so they are simply not planned (but we are always happy to review/merge if someone provides them via Pull Reuqest!).

Maybe @lmihalkovic is right: after some refactoring of the IDE internals, implementing big new features may become easy and straightforward. He has probably already did it. But until now he's just plugging on all arduino channels and keeping the code private.

Don't misunderstand me, there is nothing wrong in keeping the development private, everyone does whatever it wants with his own code. I just wanted to clarify that if we consider a particular feature useless or not worth the effort it doesn't mean that we have stopped development on the Java IDE (and again, we are always happy to reconsider our design decisions if some compelling use-cases are supplied).

[lmihalkovic]

@cmaglie Depending on their scope and the receiving end, PRs can turn into maintenance nightmares for the creator, so before making that investment, you might appreciate that i would inquire as to how the idea could be received... even if on my side I already wrote the code. As i said somewhere else, if the foundations are half-way decent, it has been my experience that a good deal of the code can be salvaged even in the face of drastic rescopings.

So i generally choose to test the concepts with code (proposing things that i don't have any sense about makes me uncomfortable, and why not make it prod-like if one goes to the trouble of writing anything at all), share a description for the user experience (more often than not as screenshots as they are more precise than words), and then carve things out if need be. So far I have seen silence, sarcasms, criticisms for the approach, and a couple of "you miss the point"s. I am not complaining. You all have production responsibilities. However it might shed some light about my not taking the time to tie a clean bow arround my changes before mentioning them.

Over the last 3 weeks i have gradually seen what might be mistaken for strong indications that IDE/desktop could quickly become Create's uggly girlfriend (not just figuratively speaking if even something as small as #4151 is not open for debate). Time will tell how much of an industrial risk it truly was for the company... but it is interesting to note that google just threw the glove on chromeos.. it looks like not all users might be ready for a brave new browser-based-world yet. right?

[lmihalkovic]

The amount of work needed to support such an optimization is tremendous

to read this about a task like this one is absolutely terrifying for the future of this application. I sincerely hope that whoever is writing Create does not consider this a tremendous amount of work.

[lmihalkovic]

A couple clarifications on how it works

• sketch folder is compressed into .INZ file with same exact internal structure
• IDE is declared to be main owner of .INZ content type (particularly important on OSX)
• single INZ can be opened directly from email or from file system. IDE works from the passed file and normally copies/compiles into temp file
• modifying source file prompts the normal save-as behavior. modified code is normally saved to sketchbook in normal expanded format

Considerations:

• IDE can ask user if she wants to copy INZ to sketchbook folder if it is opened from anywhere else. preference can be created to have a default behavior (e.g. always copy .INZ into sketchbook folder - do NOT expand)
• Link with Arduino Create ?!
• signed versus non-signed INZ
• sha1 verification
• impact on "archiver"
• tools/email sketch menu item?

AnalogReadSerial examples takes ~61K of disk space. Its zipped version is 57K. 
The amount of work needed to support such an optimization is tremendous. 
Hence such feature is useless.

the point is not and has never been the small space delta, but the improvements in usability

[matthijskooijman]

@lmihalkovic, sounds like a good way to implement this. One thing I'm missing: how would you create such an .inz file? Using some kind of export menu option? Reading on, I see you mention the "archive sketch" option, which seems to already export a zip file of the sketch.

As for signed INZ files, I'm not so sure if that's really useful. What keys would be used to sign this? Would you need to set up a GPG-style web of trust? This is probably too complex for the Arduino IDE target audience. As for SHA1 verification, I wonder if zip itself doesn't already provides checksums against transfer errors? SHA1 to protect against malicious modification would need the SHA1 to be communicated out-of-band, I do not think the IDE needs to be involved (other than perhaps offering a convenience SHA calculator).

the point is not and has never been the small space delta, but the improvements in usability

Agreed on that point.

[lmihalkovic]

@matthijskooijman core is done for sketches but still have to repair the copy-sketch-to-sketchbook-on-modify-source-file & have to test more with sketch-local libraries - see #4347 for possibly doing it with libs ( criticisms really very welcome )

My original thoughts make more sense when both sides of the equation are together ( @mbanzi )

• Arduino LLC distributes the examples as INZ in the distro (simple script during the AIDE build process)
• ACreate supports these newfangled sketches as its preferred way to expose sketches to the world (no different than what Fritzing does if you will) if/when it shares sketches out of its cloud (meaning very little work added to ACreate)
• AIDE communicates with ACreate via INZ (signing can or not be involved - this can be a add-on feature offered in the paying tier of ACreate - more can be said on this topic but this is hardly the right place)
• AIDE would support extended capabilities around INZ (again... wrong place to expand)

In no particular order

• SHA1 is indeed out-of-band
• INZ = archive + some (TBD)

[lmihalkovic]

I realize it should not be an argument, but somehow IHMO this

looks nicer to me than