Closed danielSenpaiDev closed 1 year ago
jsenko
commented
1 year ago Hi, Could you please give us more details about the features you are missing, what is your use-case, and steps to reproduce the issue with disappearing artifacts? I assume you created an external application that is modifying Registry DB tables? Is that code on GitHub?
danielSenpaiDev
commented
1 year ago Hi. Sorry for the late response. Yes, I created an external app which modifies registry DB tables. Basically I'm trying to do that the registry currently doesn't: update an artifact version / delete an artifact version. The code I have it in a private repository unfortunately. I'm going to copy here the class I'm coding (EDIT: for some reason the markdown doesn't take all my code):
`@Transactional public ArtifactProxy saveApi(String group, String api, String apiVersion, String json, String type, String contentType, Mode mode) { ArtifactProxy proxy = new ArtifactProxy(); //try Document document = Library.readDocumentFromJSONString(json); document.info.removeExtension("x-updated-on"); document.info.addExtraProperty("x-updated-on", LocalDateTime.now().toString()); // always modify the content with the current date time. json = Library.writeDocumentToJSONString(document);
// get bytes and calculate hashes to insert them later in the content table
byte[] contentBytes = json.getBytes();
String contentHash = DigestUtils.sha256Hex(contentBytes);
String canonicalHash = contentHash;
VersionContent content = new VersionContent(); // set the POJO with the info to the content table
content.setCanonicalHash(canonicalHash);
content.setContentHash(contentHash);
content.setContent(contentBytes);
Version version = new Version(); // set the POJO with the info to the versions table
version.setApi_name(api);
version.setArtifactid(api);
version.setGroupid(group);
version.setVersion(apiVersion);
version.setState("ENABLED");
version.setCreatedby("apicurio-service-account");
version.setDateCreatedOn(LocalDateTime.now());
version.setName(document.info.title);
version.setDescription(document.info.description);
Artifact artifact = new Artifact(); // set the POJO with the info to the artifacts table
artifact.setCreatedBy("apicurio-service-account");
artifact.setCreatedOn(LocalDateTime.now());
artifact.setGroupId(group);
artifact.setId(api);
//content.setContentId(registryArtifactRepository.getNextContentId());
ApiVersion storedVer = registryArtifactRepository.getArtifactVersion(api, apiVersion); //checks if the artifact/version already exists.
if (storedVer==null) { // if doesn't exist artifact nor versions, create artifact and version
long contentId = registryArtifactRepository.getNextContentId();
long globalId = registryArtifactRepository.getNextGlobalId();
Integer versionId = registryArtifactRepository.getNextVersionId(api);
versionId = (versionId==null)? 1:versionId;
version.setVersionid(versionId);
version.setContentid(contentId);
version.setGlobalid(globalId);
content.setContentId(contentId);
artifact.setType(type);
artifact.setLatest(contentId);
registryArtifactRepository.insertArtifact(artifact);
registryArtifactRepository.insertContent(content);
registryArtifactRepository.insertVersion(version);
} else { // if exists the artifact but not the version, create version
int i;
content.setContentId(storedVer.getContentId());
artifact.setLatest(storedVer.getContentId());
i=registryArtifactRepository.updateArtifact(artifact);
i=registryArtifactRepository.updateContent(content);
i=registryArtifactRepository.updateVersion(version, storedVer.getContentId());
}
proxy.setStatusCode(200);
proxy.setMessage("OK");
// } catch (Exception e) {
// proxy.setStatusCode(500);
// proxy.setMessage(e.getMessage());
// }
return proxy;
}`I'm using Apicurio Datamodels to parse the document. the methods from the object "registryArtifactRepository" executes the following sql statements:
@Insert(""" insert into artifacts (tenantid, groupid, artifactid, type, createdby, createdon, latest) values ('_', #{artifact.groupId}, #{artifact.id}, #{artifact.type}, #{artifact.createdBy}, #{artifact.createdOn}, #{artifact.latest}) """) int insertArtifact(@Param("artifact") Artifact artifact);
@Insert(""" insert into content (tenantid, contentid, canonicalhash, contenthash, content) values ('_', #{content.contentId} ,#{content.canonicalHash}, #{content.contentHash}, #{content.content}) on conflict on constraint unq_content_1 do update set canonicalhash=#{content.canonicalHash}, contenthash=#{content.contentHash}, content=#{content.content} """) int insertContent(@Param("content") VersionContent content);
@Insert("insert into versions (globalid, tenantid, groupid, artifactid, version, versionid, state, name, description, createdby, createdon, contentid) values (#{api.globalid}, '_', #{api.groupid}, #{api.artifactid}, #{api.version}, #{api.versionid}, #{api.state}, #{api.name}, #{api.description}, #{api.createdby}, #{api.dateCreatedOn}, #{api.contentid})") int insertVersion(@Param("api") Version version);
@Update("update artifacts set latest=#{api.latest}, createdon=#{api.createdOn} where groupid=#{api.groupId} and artifactid=#{api.id}") int updateArtifact(@Param("api") Artifact artifact);
@Select("update sequences set value=value+1 where name='contentId' returning value") @Options(flushCache = FlushCachePolicy.TRUE) long getNextContentId();
@Select("update sequences set value=value+1 where name='globalId' returning value") @Options(flushCache = FlushCachePolicy.TRUE) long getNextGlobalId();
@Select("select max(versionid)+1 from versions where artifactid=#{apiName}") Integer getNextVersionId(@Param("apiName") String apiName);
When I create new versions and content, it works just fine. When I update the "latest" field on the artifacts table or create a new artifact row, it's when the artifact dissapears from the registry UI.
I hope I provided enough info. Please let me know if need more details.
Kind regards.
danielSenpaiDev
commented
1 year ago UPDATE: I was checking the following code in the source code of Apicurio Registry `// Formulate the SELECT clause for the artifacts query select.append( "SELECT a.*, v.globalId, v.version, v.state, v.name, v.description, v.labels, v.properties, "
"JOIN versions v ON a.tenantId = v.tenantId AND a.latest = v.globalId "); if (joinContentTable) { select.append("JOIN content c ON v.contentId = c.contentId AND v.tenantId = c.tenantId "); }
where.append("WHERE a.tenantId = ?");`and I think I made a mistake in my code. Meanwhile the registry does a join by globalId (a.latest = v.globalid) in my side I was doing it by contentId. I updated it manually and now the registry UI shows them as usual.
I'm gonna do some more tests.
Kind regards.
danielSenpaiDev
commented
1 year ago I can confirm the join is by globalid. That's why the artifacts dissapeared from the main window. Was a misunderstanding in my side. Now is working fine.
Kind regards.
Hello. I was building a small service to work along Apicurio Registry given there is some missing features I need. The thing is I could edit and add new contents to a version, calculating the hash using the same libraries the registry uses. So far so good, I edit content and add new content/versions to an already existing artifact. The challenge comes when I want to keep the model consistent when adding a new artifact or updating the latest contentId for an existing artifact, when I do that the artifact dissapears completely from Apicurio Registry. The artifact doesn't dissapear actually, it exists in the database, it's just not showing in the UI of the registry, so I thought there is something else I need to do. I thought there was something related to the calculated hashes, but other artifacts I created using the registry UI "seem" the same way in the database as the ones created by my service. So, could anyone help me with some hints? Should I check some other classes in the source? (I already did to get the current state of my service).
Some other info I can provide: I'm just working with "artifacts", "versions", "content" and "sequences" tables. No references, labels or other entities.
Thanks in advance.