Node runtimes shouldn't be stored in SCM

fbricon commented 10 years ago

The build process could be improved to download and zip the nodejs runtimes automatically instead

angelozerr commented 10 years ago

@fbricon could you explain more your idea please.

fbricon commented 10 years ago

So in theory, that'd look like that (here the code only applies for the tern.eclipse.ide.server.nodejs.embed.win32.x86_64 plugin) :

<project  xmlns="http://maven.apache.org/POM/4.0.0"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0  http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
      <packaging>eclipse-plugin</packaging>
      <parent>
        <groupId>fr.opensagres.js</groupId>
        <artifactId>eclipse</artifactId>
        <version>0.3.0-SNAPSHOT</version>
      </parent>
      <artifactId>tern.eclipse.ide.server.nodejs.embed.win32.x86_64</artifactId>
      <properties>
      <nodejs.version>0.10.22</nodejs.version>
      <nodejs.name>node-v${nodejs.version}-win32-x86_64</nodejs.name>
      <nodejs.dir>${nodejs.name}</nodejs.dir>  
      </properties>
      <build>
        <plugins>
          <plugin>
            <groupId>org.eclipse.tycho</groupId>
            <artifactId>target-platform-configuration</artifactId>
            <version>${tycho-version}</version>
            <inherited>true</inherited>
            <configuration>
              <environments>
                <environment>
                  <os>win32</os>
                  <ws>win32</ws>
                  <arch>x86_64</arch>
                </environment>
              </environments>
            </configuration>
          </plugin>
          <plugin>
            <groupId>com.googlecode.maven-download-plugin</groupId>
            <artifactId>maven-download-plugin</artifactId>
            <executions>
              <execution>
                <id>download-nodejs</id>
                <phase>prepare-package</phase>
                <goals>
                  <goal>wget</goal>
                </goals>
            <configuration>
              <url>http://nodejs.org/dist/v${nodejs.version}/x64/node.exe</url>
              <outputDirectory>${project.build.directory}/nodejs</outputDirectory>
            </configuration>
              </execution>
            </executions>
          </plugin>
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-assembly-plugin</artifactId>
              <version>2.4</version>
              <executions>
                <execution>
                  <id>zip-nodejs</id>
                  <phase>prepare-package</phase>
                  <goals>
                    <goal>single</goal>
                  </goals>
                  <!--  The  configuration  of  the  plugin  -->
                  <configuration>
                      <finalName>${nodejs.name}</finalName>
                      <appendAssemblyId>false</appendAssemblyId>
                      <outputDirectory>${basedir}</outputDirectory>
                      <!--  Specifies  the  configuration  file  of  the  assembly  plugin  -->
                      <descriptors>
                          <descriptor>assembly.xml</descriptor>
                      </descriptors>
                  </configuration>
                </execution>
              </executions>
            </plugin>
        </plugins>
      </build>
    </project>

with an assembly.xml containing :

<assembly>
    <id>nodejs-zip</id>
    <formats>
        <format>zip</format>
    </formats>
    <fileSets>
        <fileSet>
            <directory>${project.build.directory}/nodejs</directory>
            <outputDirectory>${nodejs.dir}</outputDirectory>
            <includes>
                <include>node*</include>
            </includes>
        </fileSet>
    </fileSets>
</assembly>

When building :

[INFO] --- maven-download-plugin:1.1.0:wget (download-nodejs) @ tern.eclipse.ide.server.nodejs.embed.win32.x86_64 ---
[INFO] Got from cache: D:\Dev\maven\repository\.cache\maven-download-plugin\node.exe
[INFO]
[INFO] --- maven-assembly-plugin:2.4:single (zip-nodejs) @ tern.eclipse.ide.server.nodejs.embed.win32.x86_64 ---
[INFO] Reading assembly descriptor: assembly.xml
[INFO] Building zip: D:\Dev\GitHub\tern.java\eclipse\tern.eclipse.ide.server.nodejs.embed.win32.x86_64\node-v0.10.22-win32-x86_64.zip
[INFO]
[INFO] --- tycho-packaging-plugin:0.20.0:package-plugin (default-package-plugin) @ tern.eclipse.ide.server.nodejs.embed.win32.x86_64 ---
[INFO] Building jar: D:\Dev\GitHub\tern.java\eclipse\tern.eclipse.ide.server.nodejs.embed.win32.x86_64\target\tern.eclipse.ide.server.nodejs.embed.win32.x86_64-
0.3.0-SNAPSHOT.jar
[INFO]
[INFO] --- tycho-p2-plugin:0.20.0:p2-metadata-default (default-p2-metadata-default) @ tern.eclipse.ide.server.nodejs.embed.win32.x86_64 ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26.668 s

There are a few caveats :

Sample code needs to be refactored to a parent pom and more fine tuned (configure clean plugin to delete zip)
Properly sharing the assembly.xml file accross several modules requires extra work (generate and publish an extra jar), so you might have to either duplicate assembly.xml or try to reference a common file one directory up.
Current download plugin only checks for node.exe file name in its cache, so it'd mix x86, x64 and different version of nodejs. The alternative is to disable caching, slowing down builds
Unfortunately, there's no clean way to update the nodejs version AND the different labels in source files at the same time

angelozerr commented 10 years ago

Thank's for your explanation. @pascalleclercq could you study this issue please. Thank's!

angelozerr commented 10 years ago

@dgolovin @mickaelistria could you see this issue please. Is https://github.com/angelozerr/tern.java/pull/84 fix this issue? Thank's for your help.

gamerson commented 10 years ago

Hey Angelo, this doesn't address Fred's original issue which is the fact that the node-*.zip runtimes are still checked into git (all 20MB worth of them) which makes the git repo unnecessarily big. And also when you want to upgrade node runtimes, if you implement Fred's suggestion then the commit that "upgrades" the embedded runtimes is just a few bytes. But as it stands right now if you "upgrade" embedded runtimes, you would have to copy and upload another 20MB worth of files in a git commit.

angelozerr commented 10 years ago

Thank's @gamerson for your clarification. I have no time for the moment to try this idea.

Any contribution are welcome!

paulvi commented 10 years ago

@fbricon said

Node runtimes shouldn't be stored in SCM The build process could be improved to download and zip the nodejs runtimes automatically instead

One more option is to leverage npm Node package manager that is bundle with Node.js to install and update modules.
This however raises question if it is possible to bundle npm within Eclipse (just like was done for Node.js)

PS or possibly some npm alternatives:

http://www.jfrog.com/confluence/display/RTF/Npm+Repositories

I am sorry, I mistook runtimes for modules.

mickaelistria commented 10 years ago

@sbegaudeau is also currently working on integration of some JS tools in Eclipse, just mentioning him there so that he can follow the discussion or maybe even share with us some insights on the topic of npm in Eclipse.

paulvi commented 10 years ago

That's interesting.
please make topic of npm in Eclipse as new issue (if continued) to keep things separate

angelozerr / tern.java

Node runtimes shouldn't be stored in SCM #74