Relative paths break under a specific folder structure

s9tpepper commented 11 years ago

I ran into an issue where the relative paths in a file that is working correctly break when a new file is introduced at a completely different directory location. Because of the weirdness of the issue I have created an example project where I've replicated the issue.

The project is running Jasmine specs using node and requirejs. Running grunt will run two spec files each with one test in them. They execute without any problems.

The issue happens if I move spec/differentPackage/anotherPackage/TestFile_spec.js into the spec/differentPackage/anotherPackage/deeperPackage folder it somehow breaks the paths that are in the require() call in the source/apackage/subpackage/TestModule_spec.js file. This happens even though only TestModule_spec.js actually has a call to require. Moving TestFile_spec.js out of deeperPackage will make the paths resolve correctly again.

The error that's being thrown is coming from this line: https://github.com/s9tpepper/requirejsbug/blob/master/node_modules/jasmine-node/node_modules/requirejs/bin/r.js#L2313

For some reason the call on line 2313 thinks that the require path in TestModule_spec.js is one folder deeper, so changing require(["../../../source/apackage/subpackage/TestModule"], ...) to start with four ../ fixes it when TestFile_spec is in the deeperPackage folder. Pulling the TestFile_spec file back out of deeperPackage makes TestModule work with 3 ../ again instead of 4.

I haven't tried to go any further yet, I will try to look more during the week and will follow up with anything else I find.

https://github.com/s9tpepper/requirejsbug

jrburke commented 11 years ago

Thanks for the test repo. I could use some help though or the exact steps I need to reproduce. Something like:

Move this file to here
Change this file's require to be "this"

Part of it is I am new to jasmine, and do not know how those test files are loaded. For instance, where is the call to r.js that runs something to run the tests?

At first read, it sounds similar to this issue, but not sure if it exactly it yet: https://github.com/jrburke/r.js/issues/386

s9tpepper commented 11 years ago

Clone https://github.com/s9tpepper/requirejsbug
In Terminal run the 'grunt' command, might need to use "grunt.cmd" on Windows
You should see test results like this:

quintessa:require_bug grimlock$ grunt
path.existsSync is now called 'fs.existsSync'.
Running "lint:all" (lint) task
Lint free.

Running "jasmine_node" task
TestModule()

    getName()
        returns a name

something
    does things

Finished in 0.002 seconds
2 tests, 2 assertions, 0 failures

Done, without errors.
quintessa:require_bug grimlock$

Now go into the spec/differentPackage/anotherPackage directory and move the TestFile_spec.js file into the deeperPackage folder in the same directory that the JS file is in.
Run "grunt/grunt.cmd" again and watch the tests now fail. If you look thru the error below you can see that the file failing is Test_Module_spec.js which has not been moved and which should not be affected by moving TestFile_spec.js The output will look like below:

Running "lint:all" (lint) task
Lint free.

Running "jasmine_node" task

timers.js:103
            if (!process.listeners('uncaughtException').length) throw e;
                                                                      ^
Error: Calling node's require("../../../source/apackage/subpackage/TestModule") failed with error: Error: Cannot find module '../../../source/apackage/subpackage/TestModule'
    at req.load (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:2315:27)
    at Object.context.execCb (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:1729:33)
    at Object.Module.check (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:969:51)
    at Object.Module.enable (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:1239:22)
    at Object.Module.init (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:882:26)
    at callGetModule (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:1266:63)
    at Object.context.completeLoad (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:1623:21)
    at Function.req.load (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:2325:17)
    at Object.context.load (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:1718:21)
    at Object.Module.load (/Users/grimlock/Documents/Repositories/GitHub/require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js:928:29)
quintessa:require_bug grimlock$

I'm just now looking into the source code of these libraries for the first time so I'm trying to follow where they may be going wrong. The Jasmine tests are being run by a grunt plugin called jasmine-node, I think the key to why the error is happening is in jasmine-node's code.

The actual error being thrown is being thrown by r.js located in node_modules/jasmine-node/node_modules/requirejs/bin/r.js on line 2313.

The files that are being run through requirejs is a list put together by jasmine-node.

The grunt.js file at the root of the project has a requirejs flag set to true in the jasmine_node part of the config. That flag triggers an if statement on line 122 of node_modules/jasmine-node/lib/jasmine-node/index.js.

The list of files to execute looks to be put together by node_modules/jasmine-node/lib/jasmine-node/spec-collection.js.

Line 122 of index.js runs the node_modules/jasmine-node/lib/jasmine-node/requirejs-runner.js file. On line 73 is where it appears to loop through the paths that it finds.

I haven't been able to chase it down further, as I explore the code more I'll post more notes.

jrburke commented 11 years ago

I may have just fixed this via a report I got for r.js, and just fixed in jrburke/r.js#393, and I fixed in r.js master: https://raw.github.com/jrburke/r.js/master/dist/r.js

Can you try that version to see if it fixes the error? If not, I'll dive more into it.

s9tpepper commented 11 years ago

I'll test this again tonight after work and update this issue with my findings, thanks!

s9tpepper commented 11 years ago

I ran "npm update" in my working copy of the test repo I set up. This updated the require.js version in the jasmine_node's modules to 2.1.5. If you run it you should can check the version with "npm list". It should say:

└─┬ jasmine-node@1.2.3
  ├── coffee-script@1.6.1
  ├─┬ gaze@0.3.3
  │ ├─┬ fileset@0.1.5
  │ │ ├─┬ glob@3.1.21
  │ │ │ ├── graceful-fs@1.2.0
  │ │ │ ├── inherits@1.0.0
  │ │ │ └─┬ minimatch@0.2.11
  │ │ │   ├── lru-cache@2.2.2
  │ │ │   └── sigmund@1.0.0
  │ │ └─┬ minimatch@0.2.11
  │ │   ├── lru-cache@2.2.2
  │ │   └── sigmund@1.0.0
  │ └─┬ minimatch@0.2.11
  │   ├── lru-cache@2.2.2
  │   └── sigmund@1.0.0
  ├── jasmine-reporters@0.2.1
  ├── requirejs@2.1.5
  ├── underscore@1.4.4
  └── walkdir@0.0.5

So you can see requirejs is at 2.1.5, I opened the files just to make sure located at /require_bug/node_modules/jasmine-node/node_modules/requirejs/bin/r.js and require_bug/node_modules/jasmine-node/node_modules/requirejs/require.js

Both say version 2.1.5 after running "npm update". But I'm still seeing the same behavior. I'll try to look into it more over this upcoming weekend. I'm wondering if its actually jasmine-node that is actually the one doing something that's affecting requirejs.

s9tpepper commented 11 years ago

In r.js, when makeModuleMap() executes at the bottom of the call at line 691 there's a call to context.nameToUrl(normalizedName);

Inside nameToUrl() the code access a config.baseUrl object, that baseUrl path its uses on line 1827 is the one that is incorrect and its causing the TestModule_spec.js file to not get loaded by the require() call because the path it concatenates is incorrect. I'm not sure yet why the config object is being built with an incorrect baseUrl for that execution, I'm trying to step through the code and figure that part out next.

jrburke commented 11 years ago

I'm still on sick leave, but wanted to suggest you try the latest master version of r.js: https://raw.github.com/jrburke/r.js/master/dist/r.js

and not 2.1.5, as I did a node path fix after 2.1.5, so not part of an official release yet. That may work better.

s9tpepper commented 11 years ago

I finally tried this and the latest in master also throws the error for the same reason. The new error verified what I was seeing:

Error: Tried loading "../../../source/apackage/subpackage/TestModule" at spec/differentPackage/anotherPackage/deeperPackage/../../../source/apackage/subpackage/TestModule.js then tried node's require("../../../source/apackage/subpackage/TestModule") and it failed with error: Error: Cannot find module '../../../source/apackage/subpackage/TestModule'

It's trying to load "../../../source/apackage/subpackage/TestModule" from "spec/differentPackage/anotherPackage/deeperPackage/" which is wrong, it should be "spec/source/apackage/subpackage/".

There's a moduleMap in r.js that seems like it's somehow getting a path wrong, I'm trying to step through code to figure out where things are going wrong.

jwarkentin commented 11 years ago

@jrburke I think this is the same problem I'm having. I just created a repo you can checkout that demonstrates the issue. Clone https://github.com/jwarkentin/require-paths and set it up under some domain then go to http://yourdomain.com/require-paths/index.html and let it load. You can see a couple things logged to the console if you open up a JS console when you load the page.

The problem is with loading the underscore.string library in this case. In the loader in proj/loaders/underscore.js it uses a relative path (../lib/lodash/plugins/underscore.string) to load the underscore.string library. Unfortunately require.js is loading it relative to the base url instead of the directory it's in. If I modify the config section in main.js and move the underscore: 'proj/loaders/underscore' line down to the map section of the config then it works just fine.

There is a second problem with loading that library that is unrelated. I think it's a bug, but it'd be good for you to confirm. If you look at the AMD loader section of the underscore.string library there on line 607 it defines a module name. This seems to break loading entirely, so I swapped it out for the sake of this demo as you can see in that file. Line 608 is the original line that breaks it.

jrburke commented 11 years ago

Note for my future self: @jwarkentin's issue was a separate issue, tracked in #600.

jrburke commented 10 years ago

I just did an npm update in the requirejsbug sample project, and now I get the following output:

Running "lint:all" (lint) task
Lint free.

Running "jasmine_node" task
Failed to execute "jasmine.executeSpecsInFolder": TypeError: Cannot call method 'forEach' of undefined
    at Object.exports.load (/Users/x/git/temp/requirejsbug/node_modules/grunt-jasmine-node/node_modules/jasmine-node/lib/jasmine-node/spec-collection.js:19:13)
    at Object.jasmine.executeSpecsInFolder (/Users/x/git/temp/requirejsbug/node_modules/grunt-jasmine-node/node_modules/jasmine-node/lib/jasmine-node/index.js:98:9)
    at Object.module.exports (/Users/x/git/temp/requirejsbug/node_modules/grunt-jasmine-node/tasks/jasmine-node-task.js:93:19)
    at Object.task.registerTask.thisTask.fn (/Users/x/git/temp/requirejsbug/node_modules/grunt/lib/grunt/task.js:58:16)
    at Task.<anonymous> (/Users/x/git/temp/requirejsbug/node_modules/grunt/lib/util/task.js:343:36)
    at Task.<anonymous> (/Users/x/git/temp/requirejsbug/node_modules/grunt/lib/util/task.js:319:9)
    at Task.<anonymous> (/Users/x/git/temp/requirejsbug/node_modules/grunt/lib/util/task.js:346:11)
    at Task.<anonymous> (/Users/x/git/temp/requirejsbug/node_modules/grunt/lib/util/task.js:319:9)
    at Task.<anonymous> (/Users/x/git/temp/requirejsbug/node_modules/grunt/lib/util/task.js:346:11)
    at Task.<anonymous> (/Users/x/git/temp/requirejsbug/node_modules/grunt/lib/util/task.js:319:9)

It installed requirejs 2.1.6. So it seems to have gotten better by updates since the original report, at least it seems like now just a jasmine config issue of not finding some specs. If you feel like there still may be a requirejs issue, please let me know. However, I do not know enough about jasmine to know immediately if there is so, I will need some help tracking down. Closing for now, but we can reopen if need be.

s9tpepper commented 10 years ago

I resolved my original issues and tracked down the problem to some node modules that were newer than the versions that I was using when I originally had my project working correctly. In my case I had to create an npm-shrinkwrap.json file using the working copy that was functioning correctly so that new repository clones could install the correct npm module versions. Once I did that it worked correctly. I haven't done more research to try and figure out which npm modules exactly were the cause of the problem.

requirejs / requirejs

Relative paths break under a specific folder structure #645