Automatic self-updating of scripts

[Release]

I wanted to write an algorithm that would allow a running script to check for a new version of itself on the source server and, if found, update the copy on the current server and restart itself.

But I ran into the problem of not being able to check if the version of the script on the source server is new. I had ideas to use the date of script file change or at least the file size to determine that the file has changed. But such functions just don't exist or I couldn't find one. For example, the ls() function only allows you to get a list of files, and doesn't provide any extended information. I would like to get a function like stat(name: string, host?: string).

The only thing I came up with is to store the version number of the script in a separate file on the source server and in a loop constantly copy it to the current server and then read it to check the version. Which is kind of wrong from the point of view of the real development approach.

In the latter case, it would be good if there was a function for (streaming maybe) reading to memory without writing to a file (as scp() does now).

But maybe it's just that I'm just stupid enough and didn't see a normal way in which this can already be implemented.

P.S.

I apologize for the off-topic, but since I don't know if the problem is related to this resource, I wanted to point it out and ask if it is necessary to create a separate issue. In Steam version in the Script Editor the link to the Documentation does not work, which makes the game from there too limited, so in the documentation built into the game available API is almost not described. I had to look for the API description on the internet myself.

[catloversg]

1271 implements a new feature that will solve your problem. You can also use a workaround: use ns.read to read the script content, then compare 2 scripts.

About the in-game API documentation: We keep track of that problem in #707. In the meantime, you have to read the API documentation outside the game.

[Release]

Thank you for your reply.

You can also use a workaround: use ns.read to read the script content, then compare 2 scripts.

This variant implies infinite copying of the file from the source server to the current one with a certain interval while the script is running. Since the read() function, according to the documentation, can read a file only on the local server. I.e. you will have to constantly call the scp() function to copy the file to the local server. If there are many working scripts (as many servers), this implementation obviously does not look good from the point of view of network code, because it will cause excessive traffic. Perhaps within the game it will not matter much, but from the point of view of realism of this approach even very much.

About the in-game API documentation: We keep track of that problem in #707. In the meantime, you have to read the API documentation outside the game.

I mentioned a problem of a slightly different nature. As I wrote earlier, the problem is not the lack of complete API Documentation in the game itself, but the fact that clicking on the link in the script editor in the Steam version (which works offline) to go to the online documentation (on this resource) has no effect. While in the online version (https://bitburner-official.github.io/) it works as it should.

So the problem is only in the Steam version. So I'm not sure if this particular repository is the right one to open the issue here. Maybe there is a separate one for the Steam version.

[catloversg]

If there are many working scripts (as many servers), this implementation obviously does not look good from the point of view of network code, because it will cause excessive traffic. Perhaps within the game it will not matter much, but from the point of view of realism of this approach even very much.

That's why it's a workaround, not a proper solution. You need to wait for #1271 to get access to "FileMetadata".

clicking on the link in the script editor in the Steam version (which works offline) to go to the online documentation (on this resource) has no effect

Do you mean that clicking on the link does not do anything? If that's the case, it's a bug. Can you provide more info about that? I don't use Steam, so I cannot test it right now. However, the Steam version is just the Electron build. On my machine, when I run the Electron build and click on the link, it'll open the documentation URL in my default browser. You can join the Discord server to get help/support faster.

[d0sboots]

Since the read() function, according to the documentation, can read a file only on the local server. I.e. you will have to constantly call the scp() function to copy the file to the local server.

That's not true; you can also do the comparison on the source server, by storing the bytes of the launched file and comparing them to the current source file. This will end up being less complicated, since regardless (even with a new API) you will need something running on the source server to check the metadata and/or file contents.

You might also take a step back and look at your design for a better method. In this case, there is the fact that kill and exec work across servers, while operations like read do not. So you would be better served by having a central control process that watches for changes, and when they occur it kills the old processes, scps the new versions, and starts new copies.

[Release]

That's why it's a workaround, not a proper solution. You need to wait for #1271 to get access to "FileMetadata".

This is pretty redundant to solve the problem I mentioned, but yes it would help. Although some function getFileHash(script: string, host?: string) would be enough, similar to getScriptRam(), for example.

Do you mean that clicking on the link does not do anything?

Yes, that's exactly right. But the way the online version works, it's probably an Electron issue. I have a thought that it is only a problem with my system (I use Manjaro KDE Plasma), e.g. the default browser setting is broken. But it is not. I copied the link using debug mode from this item to a text editor that recognizes links (NotepadNext - which is analogous to Notepad++) and the link opened normally in the default browser when I clicked on it.

[catloversg]

Can you try this: API Server -> Information -> Open Extension Link (GitHub)? Does it open a link in your default browser?

[Release]

@d0sboots

That's not true; you can also do the comparison on the source server, by storing the bytes of the launched file and comparing them to the current source file.

What exactly is not true about trying to implement automatic self-updating of the script? This is how the vast majority of applications work - they request information about the new version of the application and download it if the result is positive.

Why should I check for compliance on the source server if I can't get the result of this action on the current server? Essentially, I need to check the hash of the current script on the source server against the hash of the script on the source server, passed through a parameter, let's say.

1. To do this, I need to write a script that will perform this operation and write the result to a file (I don't see any other way).
2. Then in the main script on the current server, I need to run the check script on the source server, wait for it to finish, and then copy the file with the result from the source server to the current one.
3. Then read this file to see if the script (file) has been updated or not. That is not enough that I constantly need to run the script on the source server, which will need free memory to work. And because of the cyclic nature this memory will always be needed, and if I have n running scripts on n servers, I need n times more memory. I have to copy the result file to n servers all the time.

You might also take a step back and look at your design for a better method. In this case, there is the fact that kill and exec work across servers, while operations like read do not. So you would be better served by having a central control process that watches for changes, and when they occur it kills the old processes, scps the new versions, and starts new copies.

And what makes you think that using kill() and exec() is a better method than what I want to implement? First of all, the kill() method simply kills the process no matter what stage it was in. At that time, if the process knew that it needed to terminate (e.g. for updating and restarting), it would decide (with the developer's help, of course) at what point it would be better to terminate, waiting for a certain series of operations to complete, for example. The kill() command (method) is needed only for emergency termination, it is not recommended to use it for normal workflow. You should give someone a slap on the wrist for such a thing. Next. Using the main process (script) in this case is not reasonable because the script does not know (and should not know) how many servers and what scripts are running. To do this, you have to constantly traverse all servers and check it. But (maybe I'm wrong) at the moment there is no way to get a list of servers beyond the first level. I didn't find the scan-analyze command level function. But even if it is possible, it would still require running an additional script on the source machine, which would require additional memory consumption "just because".

We have a game about real programming, not abstract automation. And such proposed approaches force you to use implementation methods, for which in real life you can get a bad mark in your resume. It is not surprising that on Steam the answer to the question “Can this game be used to teach programming?” is “No”. Client-server programming puts the overhead costs of performing operations at the forefront, and at the moment the game forces these costs to artificially increase to solve the problem. Not to mention that you suggest using the force process termination function as a normal function in a workflow.

I am not suggesting making a full-fledged implementation of the REST API (it would be enough to implement the function of obtaining a hash of a file, as I wrote above), but the methods that are currently offered are quite bad.

[Release]

@catloversg

Can you try this: API Server -> Information -> Open Extension Link (GitHub)? Does it open a link in your default browser?

Yes. I checked this and also nothing happens. There are also no entries in the debug console.

[catloversg]

In that case, the problem is in shell.openExternal. A quick check on Electron GitHub shows that this issue is Linux-specific and was reported in 2021, but it's still not fixed. We won't do anything about it.

[Release]

@catloversg I see, thank you. If this is a problem with the Electron itself and only in Linux, then of course.

[HiEv]

As a temporary workaround, to get a file hash you could simply copy blake2b_hash.js to Bitburner, then add this to the top of any script where you want to use it: import * as hash from "blake2b_hash.js";

And then in your code you could just do: let fileText = ns.read("filename.js"); let fileHash = hash.blake2bHex(fileText);

That's what I've been using to detect file changes.

[d0sboots]

Since the read() function, according to the documentation, can read a file only on the local server. I.e. you will have to constantly call the scp() function to copy the file to the local server.

That's not true; you can also do the comparison on the source server, by storing the bytes of the launched file and comparing them to the current source file.

What exactly is not true about trying to implement automatic self-updating of the script?

I was replying specifically to the part I quoted; the part that is not true is that "you will have to constantly call the scp() function to copy the file to the local server."

I outlined another way of doing things, and you replied with a bunch of additional constraints make that approach not feasible for you. Fine.

I think something you might be missing here is that, despite the veneer of how it looks, BitBurner is not a collection of independent servers running scripts. It is a single JavaScript VM containing an API and a bunch of objects. "File"s do not exist, they are really just big strings. You can communicate between servers using global variables, or module-scoped variables, or the port functions if you want to maintain the illusion. Everything is single-threaded.

This is why (for instance) there would be no point to a function that gets the hash of a file - it would be more expensive than read(), which only has to copy the reference.

Edit: To put it another way, no, this game is not about "real programming", if you take that to mean "client/server architecture programming." It absolutely is about abstract automation, although it's less abstract than most.

[Release]

@d0sboots

I was replying specifically to the part I quoted; the part that is not true is that "you will have to constantly call the scp() function to copy the file to the local server."

I outlined another way of doing things, and you replied with a bunch of additional constraints make that approach not feasible for you. Fine.

It's not like I just made up an example out of my head. The conditions I mentioned are taken from the game itself.

I think something you might be missing here is that, despite the veneer of how it looks, BitBurner is not a collection of independent servers running scripts. It is a single JavaScript VM containing an API and a bunch of objects. "File"s do not exist, they are really just big strings. You can communicate between servers using global variables, or module-scoped variables, or the port functions if you want to maintain the illusion. Everything is single-threaded.

That's understandable. But it was about letting the user write scripts the way it should be in real life.

This is why (for instance) there would be no point to a function that gets the hash of a file - it would be more expensive than read(), which only has to copy the reference.

I agree. After thinking about it (even before your reply), I came to the same conclusion. In order to reduce the cost, it would be necessary to cache the result, so that it doesn't have to be recalculated every time. But in that case, just asking for the time the file was modified would be more correct than its hash. So the mentioned development towards FileMetadata would be optimal. And if there is a function that will return only the time of change and not the whole data structure, it would be great.

Edit: To put it another way, no, this game is not about "real programming", if you take that to mean "client/server architecture programming." It absolutely is about abstract automation, although it's less abstract than most.

At the moment yes, but there's nothing stopping you from making it as close to that as you can. In this case, add a field like dateModified to the Script object, and set its value when "writing", for example in new Script(filename, code, this.hostname) and the added like modify(code) function. And then add a function to get (simplified) getLastChangeDate() { return helpers.getServer(ctx, source)?.scripts.get(filename)?.timeModified }. Or similarly for a hash, if it's still needed.

Ideally, this should be put into a separate inherited class. Then it would be possible to create quests in the game related to the date of files on the server, which would add variety.

[Release]

@HiEv Thanks for trying to help. But the problem is not how to calculate the hash, but that it can be calculated only for a local file. And for this purpose it is necessary either to copy it from the source server constantly in a loop, or to keep a script running on the source server, which will calculate the hash, compare it with the stored value, then, if necessary, make a distribution to other servers and restart them. But there are a lot of problems when there are more servers and possible scripts than a certain value.

[d0sboots]

Because the game is the way it is, generally it is best to have a single controller script in a central place (like home), and only having very simple workers running anywhere else. Some of the reasons have to do with algorithmic stuff (the "it is actually abstract automation",) and some of it is for the reasons you have run into - it is vastly more difficult to manage the kind of architecture you are trying to build.

But, everyone is entitled to have fun the way they want to. If/when the timestamps PR moves forward (it appears to be stalled right now), there will be an API-accessible interface to it. It'll still be limited to local access, though, like all file operations.

[Release]

Because the game is the way it is, generally it is best to have a single controller script in a central place (like home), and only having very simple workers running anywhere else. Some of the reasons have to do with algorithmic stuff (the "it is actually abstract automation",) and some of it is for the reasons you have run into - it is vastly more difficult to manage the kind of architecture you are trying to build.

The game as it is made, taking into account the limitations of technical capabilities. That's the thing, what I want to realize is much simpler. That's why it's used in real life. I've already given an example, all applications are updated according to this principle. The server should not care how many clients are running and how many of them need to be updated. It's the client's task to check for a new version and request the update data. There is nothing complicated about it, I described the principle above. And what you propose requires constant monitoring of information about how many servers, on how many of them a particular script should run and whether it has an up-to-date version. And if the network has several levels, and access is available only to one, then it is required to run such a script on all servers of the previous level.

But, everyone is entitled to have fun the way they want to. If/when the timestamps PR moves forward (it appears to be stalled right now), there will be an API-accessible interface to it. It'll still be limited to local access, though, like all file operations.

Getting information about the file(s) is not a file operation, at least it's not much different from getScriptRam(), getScriptLogs() or ls(), which work on remote machines. Clearly, the outcome will depend on those who control its development. I opened a thread on a particular problem. To pay attention to it or not - it is their right. As, for example, was the case with its localization - closed PR and that's it, without a sane explanation. Localization of programming and other IT stuff - it's complicated, and therefore we won't.

This is just a game - will not fit in some summary criterion, everyone can decide for themselves whether to play it or look for something else. Or even write their own, change this one since the sources are available, and the game is more for programmers. You can not play the game, but in its modification. :)

[catloversg]

As, for example, was the case with its localization - closed PR and that's it, without a sane explanation.

Which PR is that?

[Release]

Which PR is that?

https://github.com/bitburner-official/bitburner-src/pull/598 Here, as far as I understand, there was an attempt to add the possibility of localization and partial localization into Russian.

And here it was asked about the possibility of localization into German. https://github.com/danielyxie/bitburner/issues/4312 (This is if suddenly the Russian did not please in connection with known events.)

[catloversg]

hydroflame closed that PR, but it does not mean that we will never do that. I agree that hydroflame's comment is a bit short, but you can always ask for clarification (d0sboots did that). The latest attempt is in #1452. d0sboots outlined the challenges there.

About the issue in danielyxie's repository: We moved away from that repository, so nobody checks it anymore.

[Release]

@catloversg

AFAIK, none of the major/regular contributors know any languages beyond English. This means that any new features that are added won't end up translated.

This would be true if the sources were not available, and thus there would be no way for outside people to take on adding localization.

The game was fundamentally never written with translation in mind, which means there is a lot of stuff that would be hard-to-impossible to resolve. We have lots of places where we dynamically create strings by concatenation, using ternaries etc.; that would all have to be carefully broken down to be able to become translation keys. Conjugating for quantity ("1 module"/"4 modules") is a much bigger deal in some languages than it is in English; you're fortunate that that's not an issue in Japanese XD

Yes, it creates challenges, but plenty of games with localization have overcome them. As far as I know, some localization games even supported language forms for different genders. It's not an unsolvable problem.

The game involves programming, and the NS API and names of servers is and always will be in English. (hack(), n00dles, tprint(), etc.). So some English knowledge will always be needed, at a minimum.

Quoting d0sboots at the link I provided:

Wait, why does leaving some things untranslated make everything else tricky? That just means the translation is the same across all the languages for those terms, which is really easy to accomplish (you just leave it unset, and it will always take the default value, in basically all translation frameworks.)

This has been the case with programming and IT software for a long time. No one is embarrassed by it. For some reason Microsoft was able to localize Visual Studio, the same with VSCode. And not a few other development environments and other software also managed to do it.

It would be possible to give people an opportunity to implement localization, but not to add language switching until localization is ready enough. Who would want to translate the help (Documentation) for a game if the game itself remains untranslated? As far as I know it's always been the other way around. There will be localization of the game - there will be (probably) translation of documentation. Looking at the game first before looking for various guides (including walkthroughs). It's not a program to work with, and it's not the right time for people to read a thick manual before they start playing.

[catloversg]

If you want to argue about d0sboots's opinions, you need to ping them and do that in the relevant issue/PR, not here.

By the way, I think you are arguing as if we don't want to accept a PR that adds the "localization". It's the other way around. We welcome the idea (check comments from hydroflame, Snarling, and d0sboots). The problem is how to implement it properly. If you want to argue about the specific implementation details, you need to do that in the relevant PR. It's even better if you submit a PR that tackles the challenges pointed out.

[Release]

If you want to argue about d0sboots's opinions, you need to ping them and do that in the relevant issue/PR, not here.

No, I was merely responding to your comment that the problems cited are not insurmountable to simply dismiss the issue without discussion. As hydroflame did.

I'm not quite sure what PR you're expecting from me. It's impossible to cover the whole localization at once, there are not 100 lines in the game. What I would like is to be able to make gradual changes/additions for gradual localization. It is enough to revise the code for one localization and for the other 90% will be ready basis (if for example use the same react-i18next). All that remains is to translate some en.json and correct the code as needed (no more than 10%) to fit the specifics of the localized language. It would even be possible to create a separate thread for localization and make changes there with periodic merging of edits into it with the original one to keep it up to date.

As for the topic on localization, I was planning to start discussing it after this thread was closed. In order not to create a lot of discussions at once. But since there is already a topic, maybe I will join the discussion there. The main thing is that it should not be closed at once like the previous one.

[Release]

But before closing this topic, I would like to see a definite answer to the question. Can we expect some feature that will allow to receive information about script file update on a remote machine?

Since from what d0sboots said, we can understand that even FileMetadata is not planned in the near future. Although, if you want, you can make your own build, with self-added function, but this is not relevant to the topic.

[catloversg]

No, I was merely responding to your comment that the problems cited are not insurmountable to simply dismiss the issue without discussion. As hydroflame did.

I don't understand. Can you rephrase/clarify this part? Who did "dismiss the issue without discussion"? hydroflame? They did close it, but you can still discuss it with them. It may be easier to do that on Discord, though. It's real-time chat, and we are much more active there.

I'm not quite sure what PR you're expecting from me.

Both #598 and #1452. If you want to discuss with hydroflame why they closed #598 or d0sboots's opinions in #598, do that in #598. If you want to discuss how we should implement the "localization", it's best to do that in #1452. Edit: I misunderstood/misquoted this part. I mean the next part about where to discuss things.

Can we expect some feature that will allow to receive information about script file update on a remote machine?

I'll let Snarling and d0sboots answer this question.

Since from what d0sboots said, we can understand that even FileMetadata is not planned in the near future.

The PR is there right now. It's stalled because Hoekstraa has not finalized it. AFAIK, nobody objects to that PR. It'll be merged when Hoekstraa finishes it and it passes the review.

[Release]

I don't understand. Can you rephrase/clarify this part? Who did "dismiss the issue without discussion"? hydroflame? They did close it, but you can still discuss it with them. It may be easier to do that on Discord, though. It's real-time chat, and we are much more active there.

Yes. I meant that when closing this topic is not visible in the list by default due to the filter. Those. most won't see it after that unless they reset the filter or find it themselves in some other way.

Yes, you can continue to write on the topic, but there are no guarantees that they will answer. I have tried more than once to discuss an issue after closing, but usually it was simply ignored. As additional confirmation, we can take the fact that no one else wrote anything to the question (statement) of the above-mentioned d0sboots after closing. https://github.com/bitburner-official/bitburner-src/pull/598

I misunderstood/misquoted this part. I mean the next part about where to discuss things.

I understand it. I began to discuss the topic of localization here only because you drew attention to it. Initially, I gave it as an example of what can be expected from those overseeing this project. Because I've had this experience more than once. As I wrote above, I was going (a week ago) to create a topic on the issue of localization.

The PR is there right now. It's stalled because Hoekstraa has not finalized it. AFAIK, nobody objects to that PR. It'll be merged when Hoekstraa finishes it and it passes the review.

Okay then. I was focusing on this (d0sboots):

But, everyone is entitled to have fun the way they want to. If/when the timestamps PR moves forward (it appears to be stalled right now), there will be an API-accessible interface to it. It'll still be limited to local access, though, like all file operations.

I'll let Snarling and d0sboots answer this question.

Once they give a definite answer, the issue can be closed. Whatever answer they would give.

[d0sboots]

Can we expect some feature that will allow to receive information about script file update on a remote machine?

Originally, I was going to say "no". However, you raise a good point about ls - it would make sense to extend it with a new options param that lets it return detailed metadata in addition to the name.

And if the network has several levels, and access is available only to one, then it is required to run such a script on all servers of the previous level.

You're mistaken about this. The connect command can only move one "level" at a time, but api functions that take a server name can operate on any server they have the name of. You can think of it like connect only operating on local networks, while everything else routes properly.

[d0sboots]

About localization:

Yes, it creates challenges, but plenty of games with localization have overcome them. As far as I know, some localization games even supported language forms for different genders. It's not an unsolvable problem.

No one seems to be reading this part of my response in #1452:

I've done exactly this work professionally, so I know how much is involved. It's not impossible, but it would be a huge project, involving significant changes across pretty much every single file in the codebase.

If you payed me full-time to work on BitBurner, I could fix up all our string usage and get it in shape to be localized in about 1-2 person-months of work. That's about the size of the project. And no one is working on BitBurner full-time.

Games that support localization are doing it because they planned for it from the start, so they didn't make all of the many localization pitfalls that we have. Adding support incrementally is a minor cost; fixing everything after-the-fact is a huge one.

[Release]

You're mistaken about this. The connect command can only move one "level" at a time, but api functions that take a server name can operate on any server they have the name of. You can think of it like connect only operating on local networks, while everything else routes properly.

Yes, but doesn't the scan(host?: string) function return a single-level list? I mean, I won't know what's on the level above until I run the script on the next level server.

No one seems to be reading this part of my response in #1452:

I've done exactly this work professionally, so I know how much is involved. It's not impossible, but it would be a huge project, involving significant changes across pretty much every single file in the codebase.

I don't understand, what was that supposed to say? That you have to change a lot of source files with text in them? Well, that's obvious. That it can be tricky? That's obvious too. But if you don't try, nothing will happen. Just like with this game - if they hadn't started making it, it wouldn't exist.

Or is it "don't do it because it's hard".

If you payed me full-time to work on BitBurner, I could fix up all our string usage and get it in shape to be localized in about 1-2 person-months of work. That's about the size of the project. And no one is working on BitBurner full-time.

Games that support localization are doing it because they planned for it from the start, so they didn't make all of the many localization pitfalls that we have. Adding support incrementally is a minor cost; fixing everything after-the-fact is a huge one.

Oh, my God. That's the point, you don't have to do it personally. You (or whoever) posted the sources here. There are already enthusiasts who are ready to contribute to localization. Here - https://github.com/bitburner-official/bitburner-src/pull/598 that just covered after 4 PR and in the mentioned https://github.com/bitburner-official/bitburner-src/pull/1452. That is, people are proving that they are willing to spend their time for free to help add the possibility of localization.

I haven't looked at what's in the second case, but in the first earlier changes were made with universalization of localization in mind. But there they just closed it without discussing why specifically it didn't fit.

[Release]

Originally, I was going to say "no". However, you raise a good point about ls - it would make sense to extend it with a new options param that lets it return detailed metadata in addition to the name.

As for ls(), I wanted to suggest this option back in the beginning. But I rejected it because all the functions I have seen use and return values of simple types, the most complex of which is an array (in the same ls(), for example). And here you have to either combine the file name and additional information in one string, or return a complex object or json.

[d0sboots]

Yes, but doesn't the scan(host?: string) function return a single-level list? I mean, I won't know what's on the level above until I run the script on the next level server.

It needs to run with the next level server as the argument, but it doesn't need to run on the next level server. In other words, you need to do a BFS or DFS (once) to discover all the server names, but then you can just save them somewhere.

If you payed me full-time to work on BitBurner, I could fix up all our string usage and get it in shape to be localized in about 1-2 person-months of work. That's about the size of the project. And no one is working on BitBurner full-time.

Oh, my God. That's the point, you don't have to do it personally. You (or whoever) posted the sources here. There are already enthusiasts who are ready to contribute to localization. Here - #598 that just covered after 4 PR and in the mentioned #1452. That is, people are proving that they are willing to spend their time for free to help add the possibility of localization.

You're missing the point. It would take 1-2 person-months for anyone - probably longer for someone who wasn't already intimately familiar with the codebase. I wouldn't trust a new contributor to finish that kind of project. Hell, I wouldn't trust myself to finish that kind of project.

And it is important that it be finished - a partway-done refactor is worse than no refactor at all. It is this factor more than anything else - the fact that we've seen less ambitious projects fail, or land sideways due to their size - that has us convinced that this one is doomed to failure.

Edit: It would be different if one of the maintainers, or a trusted long-time contributor has this as their pet cause. That would greatly increase the confidence of the project actually finishing. But right now, the maintainers are busy just trying to keep up with PRs and releases, and all the major contributors I know are too savvy to get sucked into a huge project.

[Release]

It needs to run with the next level server as the argument, but it doesn't need to run on the next level server. In other words, you need to do a BFS or DFS (once) to discover all the server names, but then you can just save them somewhere.

Uh, thank you. Apparently I somehow didn't realize that's how it worked. Probably because of the scan command in the terminal:

Prints all immediately-available network connection. This will print a list of all servers that you can currently connect to using the 'connect' Terminal command.

You're missing the point. It would take 1-2 person-months for anyone - probably longer for someone who wasn't already intimately familiar with the codebase. I wouldn't trust a new contributor to finish that kind of project. Hell, I wouldn't trust myself to finish that kind of project.

And it is important that it be finished - a partway-done refactor is worse than no refactor at all. It is this factor more than anything else - the fact that we've seen less ambitious projects fail, or land sideways due to their size - that has us convinced that this one is doomed to failure.

That's why I suggested to create a separate branch and direct everyone who wants to take part in localization there. And let them do localization even for two years. Or at the very least, you could offer such enthusiasts to make a fork and do the localization in their own repo, and after a sane completion, you could test and make an attempt to merge to stable. The main thing is to promise people that you agree to localization and that their hard work won't end up being in vain.

The only thing that will need to be decided beforehand is the method (framework) for localization. i18n as in https://github.com/bitburner-official/bitburner-src/pull/598, or your own as in https://github.com/bitburner-official/bitburner-src/pull/1452 or some other. So that everyone who wants to get involved in localization uses the same approach, not as each new participant will want.

P.S. I hadn't suggested myself before, just because I'm not very familiar with Typescript, but even I was able to quickly change the code in StringHelperFunctions.ts convertTimeMsToTimeElapsedString() function to fit the localization:

  let res = "";
  if (days > 0) {
    res += `${days} ` + i18n.t(`common:string-helper.day.${days === 1 ? "one" : "many"}`) + ` `;
  }
  if (hours > 0 || (Settings.ShowMiddleNullTimeUnit && res != "")) {
    res += `${hours} ` + i18n.t(`common:string-helper.hour.${hours === 1 ? "one" : "many"}`) + ` `;
  }
  if (minutes > 0 || (Settings.ShowMiddleNullTimeUnit && res != "")) {
    res += `${minutes} ` + i18n.t(`common:string-helper.minute.${minutes === 1 ? "one" : "many"}`) + ` `;
  }
  res += `${seconds} ` + i18n.t(`common:string-helper.second.${!showMilli && secTruncMinutes === 1 ? "one" : "many"}`);

  return negFlag ? `-(${res})` : res;

    "string-helper": {
        "day": {
            "many": "days",
            "one": "day"
        },
        "hour": {
            "many": "hours",
            "one": "hour"
        },
        "minute": {
            "many": "minutes",
            "one": "minute"
        },
        "second": {
            "many": "seconds",
            "one": "second"
        }
    },

Edit: This is as an example to that statement:

The game was fundamentally never written with translation in mind, which means there is a lot of stuff that would be hard-to-impossible to resolve. We have lots of places where we dynamically create strings by concatenation, using ternaries etc.; that would all have to be carefully broken down to be able to become translation keys. Conjugating for quantity ("1 module"/"4 modules") is a much bigger deal in some languages than it is in English; you're fortunate that that's not an issue in Japanese XD

[d0sboots]

Your code is a perfect example of why this seems like an easier problem than it is, to someone who is not aware of the issues.

As you have implemented it, that function constructs strings via concatenation. This works in English, because the concatenation is structured so that it works in English. But it won't necessarily work in other languages - ones that need connecting words in between, for instance.

The way this actually needs to be done is to construct all the possible permutations of the phrase, and select the correct one depending on the conditions. This preserves each phrase as a single unit, so that it can be properly translated. Also, count should be passed instead of doing a ternary between "one" and "many", because there are language that have more than one plural case. But that's actually a minor detail, and easy to handle in comparison.

Also, as a side note, developing long-term on a branch (or fork) is a bad idea. It is equivalent to doing everything as one huge PR. Those are the types of PRs that land sideways, if they manage to land at all - I've had plenty of bad experiences with those, both in this codebase and elsewhere.

[Release]

As you have implemented it, that function constructs strings via concatenation. This works in English, because the concatenation is structured so that it works in English. But it won't necessarily work in other languages - ones that need connecting words in between, for instance.

Can you give me an example? To make the conversation more meaningful.

The way this actually needs to be done is to construct all the possible permutations of the phrase, and select the correct one depending on the conditions. This preserves each phrase as a single unit, so that it can be properly translated. Also, count should be passed instead of doing a ternary between "one" and "many", because there are language that have more than one plural case. But that's actually a minor detail, and easy to handle in comparison.

Yeah, I know. But, I also know that in most cases "one" and "many" are enough (I have some experience with localization too). It's not a literary essay to care about accuracy, it's just enough to make it understandable to the player and not look too clunky. Although, seen some localizations of games in Steam give the opinion that for some people it will do.

Edit: And yes, if really desired, it is possible to write a construction that will for quantitative values call a script function from the appropriate directory depending on the language chosen and return a completely correct phrase for that value. But this can be done sometime later after the main localization process is completed and if you want to achieve aesthetic perfection.

Also, as a side note, developing long-term on a branch (or fork) is a bad idea. It is equivalent to doing everything as one huge PR. Those are the types of PRs that land sideways, if they manage to land at all - I've had plenty of bad experiences with those, both in this codebase and elsewhere.

This is if you don't periodically update (merge) the localization branch from the main branch. Although now even your own fork can be updated from the main repo. "Where there's a will, there's a way."

[d0sboots]

As you have implemented it, that function constructs strings via concatenation. This works in English, because the concatenation is structured so that it works in English. But it won't necessarily work in other languages - ones that need connecting words in between, for instance.

Can you give me an example? To make the conversation more meaningful.

I found an example in German. If you believe Google Translate (a dubious proposition), "It took 5 hours 10 minutes 30 seconds." translates to "Es dauerte 5 Stunden, 10 Minuten und 30 Sekunden." I know enough German to say that translation is accurate, but not enough to speak to whether it is natural - i.e. how bad it would be to drop the conjunctions.

Honestly, English is another example. "It took 5 hours, 10 minutes and 30 seconds." is the more natural construction, but doing it the simple way with concatenation isn't so bad that we can't currently get away with it. It's like using the construction "${x} item(s)" to avoid dealing with the plural case. We're used to this in English, but it doesn't necessarily fly in other languages.

You might think this is being pedantic, and to an extent you'd be correct. However, it is precisely because I don't know 20 different languages that I am being careful like this. I If we're going to put in the time to redo the whole codebase, we're going to do it right, so that we don't have to redo it again in the future.

I do know enough about languages (and have enough experience doing this) to know about some of the major pitfalls: like how Russian and Finnish have tons of different case inflections (this kills many things based on string concatenation), or how Russian and Arabic have multiple plural forms, or how Hebrew and Arabic are right-to-left languages.

Also, as a side note, developing long-term on a branch (or fork) is a bad idea. It is equivalent to doing everything as one huge PR. Those are the types of PRs that land sideways, if they manage to land at all - I've had plenty of bad experiences with those, both in this codebase and elsewhere.

This is if you don't periodically update (merge) the localization branch from the main branch. Although now even your own fork can be updated from the main repo.

The issue is not keeping the branch up-to-date; after all, we all use forks as part of standard git development process. The issue is trying to land and enormous PR that touches everything in the whole codebase. It doesn't matter if it's up-to-date, it won't be as well tested as if it were submitted in smaller pieces. (And if/when there is a problem, touch-the-whole world changes are also a pain to rollback, particularly with intervening changes.)

[tomprince]

Because the game is the way it is, generally it is best to have a single controller script in a central place (like home), and only having very simple workers running anywhere else. Some of the reasons have to do with algorithmic stuff (the "it is actually abstract automation",) and some of it is for the reasons you have run into - it is vastly more difficult to manage the kind of architecture you are trying to build.

The game as it is made, taking into account the limitations of technical capabilities. That's the thing, what I want to realize is much simpler. That's why it's used in real life. I've already given an example, all applications are updated according to this principle. The server should not care how many clients are running and how many of them need to be updated. It's the client's task to check for a new version and request the update data. There is nothing complicated about it, I described the principle above.

If you want to implement something like this, you could implement an update server that runs on home (or wherever) that serves info about the files available there (using netscript ports for example).

[Release]

I found an example in German. If you believe Google Translate (a dubious proposition), "It took 5 hours 10 minutes 30 seconds." translates to "Es dauerte 5 Stunden, 10 Minuten und 30 Sekunden." I know enough German to say that translation is accurate, but not enough to speak to whether it is natural - i.e. how bad it would be to drop the conjunctions.

When I asked for an example, I meant an example of localization in a game for some language. I.e. if you chose German as the language, I would like to see localizations for German and, preferably, for the game to be comparable in level (i.e. not AAA level).

Honestly, English is another example. "It took 5 hours, 10 minutes and 30 seconds." is the more natural construction, but doing it the simple way with concatenation isn't so bad that we can't currently get away with it. It's like using the construction "${x} item(s)" to avoid dealing with the plural case. We're used to this in English, but it doesn't necessarily fly in other languages.

Yes I too am aware of multiple forms for a certain interval of values. But, as I said above, we should consider localization not as a translation of a fiction text, but as it is accepted (in most cases) at the moment when localizing games of the appropriate level. I know, for example, that Polish, Ukrainian and Russian use interval declensions: "1 minuta, 2 minuty, 5 minut" (Polish), "1 хвилина, 2 хвилини, 5 хвилин" (Ukrainian), "1 минута, 2 минуты, 5 минут" (Russian). But I also know that localization often uses a simplified form for the singular "minuta" for the plural "minut" (for the other languages mentioned, so too).

And even if you really want more, there are already solutions for these problems - https://www.i18next.com/translation-function/plurals.

You might think this is being pedantic, and to an extent you'd be correct. However, it is precisely because I don't know 20 different languages that I am being careful like this. I If we're going to put in the time to redo the whole codebase, we're going to do it right, so that we don't have to redo it again in the future.

You have the sources lying in the public domain with version control system, which allows you to change this or that code at any time, to create separate branches when developing some additional function, and most importantly to use the labor of enthusiasts who know better than you in a particular aspect, for example in translation into a particular language (in the same Japanese). Why don't you let them decide whether that particular language needs perfection or a simplified version is enough?

It's not pedantry, it's perfectionism. Do you really think that the absence of localization is better than its presence, even if it is not 100% correct in relation to a particular language? From our conversation, I got the impression that I was trying to persuade a girl to go on a date, rather than offering to make the game more convenient and friendly for more potential players.

The issue is not keeping the branch up-to-date; after all, we all use forks as part of standard git development process. The issue is trying to land and enormous PR that touches everything in the whole codebase. It doesn't matter if it's up-to-date, it won't be as well tested as if it were submitted in smaller pieces. (And if/when there is a problem, touch-the-whole world changes are also a pain to rollback, particularly with intervening changes.)

How do thousands and more developers in the same number of projects, and not only in localization development, cope with it? How did those (enthusiasts, among others) who made localization for ready-made games, which were not developed with localization in mind at all, survive? I have already said, if a person wants, he acts and finds an opportunity, if he does not want, he is always looking for reasons not to do and excuses.

I'm sick of saying obvious things about how your case is not unique in the slightest and the whole world has dealt with it before and is dealing with it now. Therefore, I propose to end this pointless conversation, because we do not discuss this or that specific problem that arose during localization, but waste time on meaningless arguments about whether localization is necessary at all, if there are difficulties.

[Release]

If you want to implement something like this, you could implement an update server that runs on home (or wherever) that serves info about the files available there (using netscript ports for example).

To be honest, I haven't dealt with ports yet. If you are familiar with them, can you give a simplified example for this task?

[d0sboots]

I'm sick of saying obvious things about how your case is not unique in the slightest and the whole world has dealt with it before and is dealing with it now. Therefore, I propose to end this pointless conversation, because we do not discuss this or that specific problem that arose during localization, but waste time on meaningless arguments about whether localization is necessary at all, if there are difficulties.

That is fine. I only kept responding because you kept asking more questions and/or raising more points. FWIW, I think localization is probably the single most impactful feature we could add, in terms of bringing more people to the game. However, even with that in mind I think the risks are too high, such that the cost/benefit isn't there. You disagree, and that's OK.

[catloversg]

@d0sboots We can close this issue. OP knows how to solve their original problem.