CoARCHI unable to clone the model with size >550MB hosted in GitLab #200

Closed rmsousa27 closed 1 year ago

Hello,

I've a model that have more than 550 MB and through ARCHI, I'm unable to clone. Instead, the clone process finish by creating a "new model".

I've tested the clone process using the Git Bash and it has complete successfully.

Version of Archi, Operating System

ARCHI version 4.10.0 Build 202209150946 , Microsoft Windows [Version 10.0.22621.819]

Version of this plug-in

0.8.6.202210061525

Any other plug-ins installed?

None

Expected Behaviour

It should clone the model in the repo

Actual Behaviour

It won't clone the model and it ends to create a new model.

Steps to Reproduce the Behaviour

Have a model with more than 550MB hosted in a Gitlab Repo;
Select "Add Remote Model"
Insert the HTTPS url (Gitlab), username and password;
Process initiate and finish with no error log but without clone the remote model.

Thanks for the report. In order to diagnose I need to reproduce this and need a repository of that size. Do you have an example repository I can test?

Hi, great to ear you back so soon. Unfortunately, I don't have any example repositories other than the one that is giving problems, and I can't share this one because of the sensitive data in it. Is there any other way that you can test this?

Is there any other way that you can test this?

Unfortunately, no. I would need a repository of that size, and I don't have one. I understand your concern about sensitive data.

There are two stages to the clone operation - git clone (done by the JGit library) and then converting all of the small xml files into an Archi model. I need to figure out which stage is failing.

Hi,

Some questions:

Did this happen suddenly or did you (or your colleagues) already faced issues from time to time?
Do you know what makes your repo so large? Is it because of the size of the model and a long history, or is it because there's a lot of images (real images added to canvas view or used as custom icons)?
Can you provide the error log which could help understand what happens?
Could you try to create a filtered version (with shorter history) of your repo, host it on Gitlab and try to import that version?

FWIW, me and my colleagues work on repos which are 4 years old and reach 200-250MB and we never faced any issues.

Is there any other way that you can test this?

Unfortunately, no. I would need a repository of that size, and I don't have one. I understand your concern about sensitive data.

There are two stages to the clone operation - git clone (done by the JGit library) and then converting all of the small xml files into an Archi model. I need to figure out which stage is failing.

screenshot of XML files

I think I found the issue. I notice that some XML files have a size of 110MB each. So I open one of this file and find out that the Documentation tag content have more than 113 Million chars and it is all rubish.

screenshot of XML file content

And I just confirmed that all of them, have rubish in the documentation tag.

I will do a clean on it and try again and see if the reported issue still stand.

Hi,

Giving an update on the matter, I have deleted all the junk text from the xml that was within the "Documentation" tag and it seems to have solved the problem due to the huge reduction in xml file sizes that reside in the repo. What I noticed is that the text was repeated over and over again as you can see in the image I shared in my previous comment, which I found odd. I can't understand where this content came from?

Another observation is the structure and folder names of the model are very large and when I try to merge the branches using Windows OS, I get an error that the path is too long and the cloning process is interrupted.

archimatetool / archi-modelrepository-plugin