[Feature Request] Support Index Map ("sections")

[vonBrax]

Given two source TS files:

src/a.ts

export default class A {}

src/b.ts

export default class B {}

The files are compiled from TS to JS, which also includes some babel transforms, generating each one a JS file and its accompanying source map file:

dist/a.js

"use strict";sap.ui.define([],function(){class s{}return s});
//# sourceMappingURL=a.js.map

dist/a.js.map

{
  "version":3,
  "file":"a.js",
  "names":["A"],
  "sources":["../src/a.ts"],
  "sourcesContent":["export default class A {}\n"],
  "mappings":"yCAAe,MAAMA,GAAI,OAAAA,CAAA",
  "ignoreList":[]
}

dist/b.js

"use strict";sap.ui.define([],function(){class s{}return s});
//# sourceMappingURL=b.js.map

dist/b.js.map

{
  "version":3,
  "file":"b.js",
  "names":["B"],
  "sources":["../src/b.ts"],
  "sourcesContent":["export default class B {}\n"],
  "mappings":"yCAAe,MAAMA,GAAI,OAAAA,CAAA",
  "ignoreList":[]
}

The JS files are then concatenated together to create a single bundle file, which generates:

dist/bundle.js

//@ui5-bundle @admin-center/core/bundle.js
sap.ui.require.preload({
  "@admin-center/core/a.js":function(){
    "use strict";sap.ui.define([],function(){class s{}return s});
  },
  "@admin-center/core/b.js":function(){
    "use strict";sap.ui.define([],function(){class s{}return s});
  }
});
//# sourceMappingURL=bundle.js.map

dist/bundle.js.map

{
  "version":3,
  "file":"bundle.js",
  "sections":[
    {
      "offset":{"line":1,"column":0},
      "map":{
        "version":3,
        "names":[],
        "sources":["bundle.js?bundle-code-0"],
        "mappings":"AAAA;AACA",
        "sourcesContent":["sap.ui.require.preload({\n"],
        "sourceRoot":""
      }
    },
    {
      "offset":{"line":3,"column":0},
      "map":{
        "version":3,
        "file":"a.js",
        "names":["A"],
        "sources":["../src/a.ts"],
        "sourcesContent":["export default class A {}\n"],
        "mappings":"AAAA,yCAAe,MAAMA,GAAI,OAAAA,CAAA",
        "ignoreList":[],
        "sourceRoot":""
      }
    },
    {
      "offset":{"line":6,"column":0},
      "map":{
        "version":3,
        "file":"b.js",
        "names":["B"],
        "sources":["../src/b.ts"],
        "sourcesContent":["export default class B {}\n"],
        "mappings":"AAAA,yCAAe,MAAMA,GAAI,OAAAA,CAAA",
        "ignoreList":[],
        "sourceRoot":""
      }
    }
  ]
}

In the browser, the bundle.js file is served instead of the individual JS files. In the devtools, the sources from bundle.js are correctly mapped to the original TS files. This is also true when using the Source Map Visualization.

But when trying to collect and analyse the coverage data through playwright together with monocart-reporter , monocart-coverage-reports seems to be unable to understand this source map syntax and treats bundle.js as a src file, instead of a dist file, keeping it in the final coverage report and showing coverage data for bundle.js instead of extracting the source files from it to show the individual files coverage data.

The Index Map specs can be found in the TC39 repo or in the tc39.es website.

Are the any plans to support the index map syntax? Or any workarounds that I could do in the meantime to retrieve the coverage data for the individual files inside the bundle?

I did try to play around with the collect-source-maps.js file to achieve that, but couldn't wrap my head around on how to extract the coverage data from a index map source map file.

We are using the OpenUI5 frontend framework together with the UI5 Tooling library, responsible for the bundling and bundle source maps generation (amonst other things).

[cenfun]

Yes, MCR does not support SectionedSourceMaps yet.

I'm not very familiar with it, but it would be a good feature if MCR could support it, I found an example implementation here: https://github.com/jridgewell/trace-mapping?#sectionedsourcemaps

[vonBrax]

Sounds great! I'll check the link and see if I can come up with something here

[cenfun]

I haven't found a build tool(such as webpack, esbuild) that can generate SectionedSourceMaps yet. Could you please help to create an example repo that can simply generate SectionedSourceMaps? Something like:

• git clone repo-url
• npm install
• npm run build

The dist file and sectioned sourcemap file will be generated. Thanks.

[vonBrax]

You can clone the OpenUI5 Sample App repo:

git clone https://github.com/SAP/openui5-sample-app
cd openui5-sample-app
npm install
npm run build

The build output will be in the dist folder and the file with the sectioned sourcemap is the dist/Component-preload.js file

[cenfun]

I've created a patch to support SectionedSourceMaps, see commit. However, I don't have more test case to test it. Could you help with that? For example:

• I have few source files like: a.js. b.js, c.js
• npm run build
• it genereated bundle.js and bundle.js.map (SectionedSourceMaps)

Which build tool can do this? How to generate a bundle file with SectionedSourceMaps easily?

[vonBrax]

I've been trying to find out other tools that generate sourcemaps in the index map format, but the only other one that I could find is react-native, as mentioned in this sentry issue.

I also tried to play around with rollup using different configs and plugins, but I could never generate a sectioned sourcemap.

From the top of my head, the two options that we have would be:

1. Use the openui5 sample app setup:

   • Pro: Generates sectioned sourcemaps
   • Con: A lot of boilerplate and not easy to test due to the runtime dependency on the openui5 framework

2. Use rollup with multiple entry points and concatenate the output similar to what rollup plugin-multi-entry does, but also generating a sectioned source map in the process.

   • Pro: easier setup, easier to test
   • Con: needs a custom script to generate the sectioned maps

I think in the end the second option seems better, so I'll try to see if I can create a script to generate a sectioned source map from a set of source map files.

Let me know if you have any other ideas

Ah btw, not sure if relevant anymore, but I also found this library that takes a sectioned source map and flattens it to the "normal" source map format: flatten-source-map. Maybe would be worth taking a look into it?

Thank you for taking the time to look into this issue!

[vonBrax]

OK, so I came up with this setup

It's basically a rollup project that:

1. Compiles your source files into the dist folder
2. Uses the rollup multi-entry plugin to generate a single bundle from all compiled files
3. Run a custom script to generate a sectioned source map from the available source maps

I'm not really familiar with source maps, so I can't promise that this script does what it should do. But at least in the source map visualization tool everything seems to match.

You can clone the repo and then just add any source files into the src folder. Then run npm run build to generate the bundle file along with its sectioned source map.

I hope that can help you create some more test cases.

I'm gonna pull your changes now to test it against my app and see if I can find any issues with it.

[cenfun]

Thanks for your repo sample-index-map-gen But I don't think it's what I want.

• after rollup -c, it generated sourcemaps for each source files, and it also merged all into bundle, but the bundle.js.map is not an indexed sourcemap.
• node scripts/sourcemap.js it seems that it is converting a regular sourcemap to an indexed one. not sure the conversion is correct.

In fact, What I'm looking for is not a build tool like rollup, but some tool that works just like your scripts/sourcemap.js (It should not be converting an existing regular sourcemap, but rather concatenating multiple regular sourcemaps into one). Unfortunately, I've been searching for a long time, but I haven't found it yet. It seems that indexed sourcemap has not been accepted by popular tools. see similar issue: https://github.com/evanw/esbuild/issues/3439

Indeed, OpenUI5 Sample App can generated some indexed sourcemaps, but those files cannot be directly used as test cases, and I don't know how to use OpenUI5 to generate the indexed sourcemaps what I want. Any idea?

[cenfun]

I thought if the indexed sourcemap is not supported by most of build tools, how about we fix this issue in OpenUI5, It said, OpenUI5 can generate the regular sourcemaps instead of indexed sourcemaps, just like what rollup does.

[vonBrax]

Ok, so I updated the sample-index-map-gen repository.

It is now basically a repository where you can add some javascript files and then it uses the OpenUI5 setup (through UI5-tooling) to create a bundle containing all files that you add to the webapp folder.

The bundle.js file and its sourcemap will be located under dist/resources after you run npm run build.

Since the bundle.js file is a simple concatenation of all source files, it has no dependencies on the UI5 framework, so you can just use the bundle file direct in the browser for testing.

I could open an issue in OpenUI5 to see if they would be willing to change how they generate sourcemaps, but tbh I don't think that this is something they would really want to do, nor do I see that happening sometime in the near future.

Furthermore, I already tested your changes against my codebase that uses the sectioned maps and it looks really good! I still need to take a deeper look at it, but it seems to be working great so far.

Let me know if the new repo setup does what you need and also if you need help adding tests for the sectioned maps.

[cenfun]

Thanks for the update which is very close to what I want. However, the sourcemap is not complete, it does not include the sourcesContent.

{"version":3,"file":"a.js","names":["A","console","log"],"sources":["a-dbg.js"],"mappings":"AAAA,SAASA,IACPC,QAAQC,IAAI,WACd","ignoreList":[]}

And the sources should be ["webapp/a.js"], I don't know why is a-dbg.js. Is there any way to fix the sources and sourcesContent?

[vonBrax]

Hey, sorry for that, I didn't realize these issues. I made some changes to the repo and now it's correctly generating the sourcemaps.

Regarding the -dbg.js files, ignore that, it's fixed. They use this files for debugging into source code (as opposed to transpiled files), before sourcemaps existed I guess. Why they still generate those files I have no idea.

I hope this time it will work!

[cenfun]

Except the simple a.js and b.js, I have added some other common test cases, and it seems works well. Thanks for your help and test cases repo. Next, I need to refactor the current code for better performance, then there should be a patch version release, It won't take long time.

[cenfun]

Please try version monocart-coverage-reports@2.9.2

[vonBrax]

Thank you @cenfun!

I've already installed the latest version, and as far as I can tell, everything seems to be working, and now I'm getting the coverage for all individual files.

Thank you for you work in here!

I'm closing this issue as completed.