Add support for a .soignore file (like a .gitignore, .dockerignore file)

[rbeckham]

It would be nice if there was a way I could tell source orbit to ignore certain dependencies.

For example, in my project I am developing a local project, but I am using some tables that are used across the entire company. (ie. Our company order detail file) This table is used in probably every project i create. At the moment, there is no way I can bring in the source to build this table in my local project as it is also being used everywhere in our legacy environment.

I'm not sure this is a good idea? :-) but I wanted to suggest it and see what you thought. I'm sure I'm not the only one stuck in hybrid environment where we have 1000s of objects in legacy mode still but trying to migrate everything new to local projects.

Or am I thinking about this the wrong way?? maybe I"m missing something.

[worksofliam]

So it sounds like to me, you want to include objects in your repo so the references can be found, but you don't want them to be included in the build files?

For sure we can do that, but I don't think it will be via a dot file. I am thinking some project setting which says "this directory is for references" only.

Let me think about it, but does that sound about right?

[rbeckham]

I am thinking some project setting which says this directory is for references only

My only concern is these objects (physical files) live in source members and are compiled by legacy programmers using DDS the "the old fashioned way". (SEU, PDM) so there would be no IFS directory to reference. UNLESS I could use a /QSYS.LIB/.... path. That might actually solve a lot of my problems, is if you would allow me to reference source members still in libraries and IFS directories too that would not get built. That would give me the flexibility to continue writing newer code using new methods but allow me to tap into the legacy world.

I feel like this has turned more into a discussion now. I apologize for that. Please feel free to close this at any time! I know you got a lot to do. I just know that the problem I'm trying to describe is the biggest hurdle I have in trying to move our company towards local development. I have to somehow live in both worlds (legacy source members, and local development) and I feel like alot of customers will be in my position.

[worksofliam]

@rbeckham No worries - discussion is great!

Yeah, we need a list of objects (that exist elsewhere, like in a library) that can be found by the Source Orbit resolver basically.

So I think we need to do this:

• Allow the user to mark certain directories as 'references' only so that they are not included in the build files, but can be resolved
• Perhaps instead of a directory, it could be a simple text file that contains a list of objects that exist elsewhere (like in QSYS) which make Source Orbit think all those objects exist locally so they can be resolved by Source Orbit.

I understand your issue I think. You want to use git for new source/objects on top of existing objects in QSYS.

[edmundreinhardt]

I agree that it should be a list of objects, and that for BOB or Makefile these dependencies are after the '|' so that there is no attempt to build them.

[worksofliam]

@rbeckham I am going to add two new optional properties to the root iproj.json.

• referenceDirectory which will use sources in that directory as references only and will not be in the built file outputs
• referenceList which will point to a file with a list of objects (CUSORD.FILE, FAQ400.PGM, etc) which will be created as references only and not in the build files.

Hope that will suffice.

[edmundreinhardt]

Is there a usecase for referenceDirectory? I don't see one and I don't think Ryan is requesting that. If we can include the source in the project than it is under this source control. I think the issue is more with external objects. I can see a request for reference library but that would be slightly more difficult to implement. Basically any objects in that library become referenceList object unless the object can be created from source in the project.

So I would propose we only support referenceList Also referenceList is not interpreted by BOB but only by those generating BOB/makefiles

[rbeckham]

Initial thoughts is I think that is perfect! That gives me the flexibility to leave legacy objects where they are. Our legacy developers can still work as usual and our modern git developers can continue on!

I'm not sure we need a reference Directory now....and also if we include reference directory that points to source members in /QSYS.LIB/xxx then Source Object would become dependent on Code For i being connected to our system...not sure that is optimal.

Also having them still included in the references will be great. I'm thinking I can run the so -bf json command and still have all dependencies listed! I can take that json file and also import those references into our legacy x-ref application (Hawkeye) so we have visibility everywhere! Exciting times.

[worksofliam]

also if we include reference directory that points to source members in /QSYS.LIB/xxx then Source Object would become dependent on Code For i being connected to our system...not sure that is optimal.

Correct, that isn't what I want to do either! The best part about SO is the fact is can run on multiple systems (like GitHub Actions) to create the builds, before deploying to IBM i. My idea behind the reference directory is so you can add source code to the repo that is basically ignored and only used to build a list of objects (and not scanned)

Let me work on a PR in the near term and post back here.

[rbeckham]

The best part about SO is the fact is can run on multiple systems (like GitHub Actions) Since you mentioned that it runs on Multiple systems.....

Does SO work on windows? Have you successfully run commands on a Windows system? I have tried for 3 days now to get SO to work on my local windows machine and I can't get it to work. I can install using npm but when i run any command it just does nothing. It prints no text to terminal. I even did a so -h and it would not even print the help text. I just want to confirm it runs on windows before I start rebuilding my system.

[worksofliam]

@rbeckham Yes, it does run on Windows. We can prove this as we have test cases inside of this repo that are run on Windows automatically.

[edmundreinhardt]

also if we include reference directory that points to source members in /QSYS.LIB/xxx then Source Object would become dependent on Code For i being connected to our system...not sure that is optimal.

Correct, that isn't what I want to do either! The best part about SO is the fact is can run on multiple systems (like GitHub Actions) to create the builds, before deploying to IBM i. My idea behind the reference directory is so you can add source code to the repo that is basically ignored and only used to build a list of objects (and not scanned)

Let me work on a PR in the near term and post back here.

Can't we consider any object in the referenceList as terminals. I.e. we don't need to scan any further. This would work for references to DB tables and view, or program calls. Hmm I am starting to realize what you are thinking. What about references to procedures, or subcomponents. Perhaps the externals can simply be a JSON with the object name and array of referenceable elements in it. This can be provideed manually, scanned from objects or scanned from source. This can in a single .externalReferences file or directory by that name.

[rbeckham]

My idea behind the reference directory is so you can add source code to the repo that is basically ignored and only used to build a list of objects (and not scanned

If I understand correctly, my only concern with reference directory is the code may become stale if I'm copying code in that directory. When legacy developer updates the source on one of these objects on the legacy side, now my copied code in the local project is outdated. I would think just a list of object names to ignore would be sufficient, but you may be seeing things I'm not.

[edmundreinhardt]

I agree Ryan, but at some point we need to scan the Object/source and extract the referencable items in it (like subprocedures, record formats, fields, etc)

That is why I am proposing an intermediate format that only has this information, rather than the full text. This information can be computed and synchronized at another time when there is a live connection for example.

Then all of the source orbit functions can be done in a CI/CD pipeline without necessarily access to a connection

[rbeckham]

@edmundreinhardt I see what you are saying now. That makes sense.

[worksofliam]

@rbeckham I am going to work on this today. I am going to create something like .sorefs which will include a list of objects that will be included so when they are referred to they can be found.

PR incoming.

[edmundreinhardt]

@worksofliam What do you think of including references of the form mypgm mypgm-procedureCall mypgm-proc2 mydb mydb-recfmt mydb-recfmt-fld1

[worksofliam]

@edmundreinhardt Good idea, I'd like to meet with you today to discuss this further if that is ok with you. I will reach out to you.

[worksofliam]

The .sorefs file will look something like this:

TABLE.FILE
OTHER.FILE
ABCD.PGM
COOL.MODULE
  coolthing
  otherprocedure
THINGY.SRVPGM
  exportA
  exportB

[rbeckham]

Thanks for the attention to this guys. Im looking forward to trying it out once you get it developed!

[worksofliam]

@rbeckham The CLI will have support for the .objrefs file. I have merged the PR.

Here is the page to the docs. I will do a release of 0.9.0 which will support this feature from the CLI.

[rbeckham]

@worksofliam I had a chance to try this out today. I like what I see! I love the fact it still lists these objects as dependencies when i create the json reference file. I should be able to use this json file and import that info into our legacy xref application (hawkeye) and that way our legacy developers can see the objects the code in GitHub is touching since none of my new modern code is stored on the IBM i.