rdfhdt / hdt-java

HDT Java library and tools.
Other
94 stars 69 forks source link

K-HDTCat #181

Closed ate47 closed 1 year ago

ate47 commented 1 year ago

Hello!

In this PR, I've implemented a version of HDTCat working whatever the input count of HDTs (for MS/4S Dictionaries), it follows the same algorithm as HDTCat_let's_make_HDT_scale, but with multiple HDT input, except that the streams from each HDT are merged to behave like one HDT

I've added it in HDTCatTree with the option loader.cattree.kcat to set the number of HDT to merge at the same time

3 options are here to describe the cat:

API

I've added all the options in the HDTOptionsKeys class

A new method was added in HDTManager

public static HDT catHDT(List<String> hdtFileNames, HDTOptions hdtFormat, ProgressListener listener) throws IOException

Using the old method will use the legacy algorithm

CLI

I've added the BATCH version of hdtCat with the hdtCat.bat

I've added the -kcat argument to HDTCat to allow to use the new version of HDTCat with 2 HDTs, but now we can specify the number of HDTs we want

With hdtVerify, I've removed the limit of 1 hdt, I've added 2 arguments:

Core

The implementation of HDTCat, with tests

D063520 commented 1 year ago

yea! nice!!!