Question: How to generate FAI files and TBI files

[Lys9527]

I recently used htsjdk 3.0.5 to complete my own project, I would like to ask how htsjdk needs to be used to generate fai files and tbi files, thank you

[lindenb]

FAI:

https://github.com/samtools/htsjdk/blob/master/src/main/java/htsjdk/samtools/reference/FastaSequenceIndexCreator.java

TBI:

private static int copyTo(final Path invcf,final Path outvcfpath) throws IOException {
    try(VCFReader vcfReader1  = new VCFFileReader(invcf,false)) {
        final VCFHeader header1 = vcfReader1.getHeader();
        final VariantContextWriterBuilder vcb = new VariantContextWriterBuilder();
            final SAMSequenceDictionary dict1 = header1.getSequenceDictionary();
            vcb.setReferenceDictionary(header1.getSequenceDictionary());
            vcb.setIndexCreator(new TabixIndexCreator(dict1, TabixFormat.VCF));
            vcb.setOption(Options.INDEX_ON_THE_FLY);
            vcb.setOutputPath(outvcfpath);
            vcb.setCreateMD5(false);

            try(final VariantContextWriter w = vcb. build()) {
                w.writeHeader(header1);
                try(final CloseableIterator<VariantContext> iter1 = vcfReader1.iterator()) {
                    while(iter1.hasNext()) {
                        w.add(iter1.next());
                        }
                    }
                }
        }

    return count;
    }

[Lys9527]

FAI:

https://github.com/samtools/htsjdk/blob/master/src/main/java/htsjdk/samtools/reference/FastaSequenceIndexCreator.java

TBI:

private static int copyTo(final Path invcf,final Path outvcfpath) throws IOException {
  try(VCFReader vcfReader1  = new VCFFileReader(invcf,false)) {
      final VCFHeader header1 = vcfReader1.getHeader();
      final VariantContextWriterBuilder vcb = new VariantContextWriterBuilder();
          final SAMSequenceDictionary dict1 = header1.getSequenceDictionary();
          vcb.setReferenceDictionary(header1.getSequenceDictionary());
          vcb.setIndexCreator(new TabixIndexCreator(dict1, TabixFormat.VCF));
          vcb.setOption(Options.INDEX_ON_THE_FLY);
          vcb.setOutputPath(outvcfpath);
          vcb.setCreateMD5(false);

          try(final VariantContextWriter w = vcb. build()) {
              w.writeHeader(header1);
              try(final CloseableIterator<VariantContext> iter1 = vcfReader1.iterator()) {
                  while(iter1.hasNext()) {
                      w.add(iter1.next());
                      }
                  }
              }
      }

  return count;
  }

Thank you very much for your answer, I have understood the generation of fai file, for tai file, do you need to limit the format of the input file here? For example, gff file, sorry I am not learning this major, but I need to use this function in my project, so the problem is a little rude, please forgive me

[lindenb]

for TBI +BED, you can use

            TabixIndexCreator indexCreator=new TabixIndexCreator(TabixFormat.BED);
            writer = new BlockCompressedOutputStream(this.outputFile);

                long filePosition= writer.getFilePointer();
                while(in.hasNext())
                    {
                    String line = in.next();
                    BedLine bed = this.bedCodec.decode(line);
                    if(bed==null) continue;
                    writer.write(line.getBytes());
                    writer.write('\n');
                    indexCreator.addFeature(bed, filePosition);
                    filePosition = writer.getFilePointer();
                    }
                }
            writer.flush();
            Index index = indexCreator.finalizeIndex(writer.getFilePointer());

it's the same idea for GFF but used a gff3 codec: https://www.javadoc.io/static/com.github.samtools/htsjdk/2.24.1/htsjdk/tribble/gff/Gff3Codec.html

[Lys9527]

for TBI +BED, you can use

          TabixIndexCreator indexCreator=new TabixIndexCreator(TabixFormat.BED);
          writer = new BlockCompressedOutputStream(this.outputFile);

              long filePosition= writer.getFilePointer();
              while(in.hasNext())
                  {
                  String line = in.next();
                  BedLine bed = this.bedCodec.decode(line);
                  if(bed==null) continue;
                  writer.write(line.getBytes());
                  writer.write('\n');
                  indexCreator.addFeature(bed, filePosition);
                  filePosition = writer.getFilePointer();
                  }
              }
          writer.flush();
          Index index = indexCreator.finalizeIndex(writer.getFilePointer());

it's the same idea for GFF but used a gff3 codec: https://www.javadoc.io/static/com.github.samtools/htsjdk/2.24.1/htsjdk/tribble/gff/Gff3Codec.html

Thank you very much for your detailed answer. If you can give me an example, for example, my file is /path/to/SCHZ.rRNA.gff.gz, how do I need to generate the “/path/to/SCHZ.rRNA.gff.gz.tbi” file?Sorry I'm new to the java building website project, very unproficient, bothering you！

[gokalpcelik]

Using codecs and IndexFactory should do fine I guess. Reiterating over the actual file may not be necessary

Here is my code from my repository

final VCFCodec codec = new VCFCodec();

        if (this.VCFFile.getAbsolutePath().endsWith(FileExtensions.VCF)) {
            idxpath = this.VCFFile.getAbsolutePath() + FileExtensions.TRIBBLE_INDEX;

            try {
                final Index idx = IndexFactory.createDynamicIndex(this.VCFFile, codec,
                        IndexBalanceApproach.FOR_SEEK_TIME);
                idx.write(new File(idxpath));
            } catch (final Exception e) {
                if (e instanceof UnsortedFileException) {
                    OverSeer.log(this.getClass().getSimpleName(),
                            "VCF File is not sorted. Sorting function is currently not implemented. Please sort your vcf using a proper tool.",
                            OverSeer.ERROR);
                } else {
                    e.printStackTrace();
                }

            }

        } else if (this.VCFFile.getAbsolutePath().endsWith(FileExtensions.COMPRESSED_VCF)) {
            idxpath = this.VCFFile.getAbsolutePath() + FileExtensions.TABIX_INDEX;

            try {
                final Index idx = IndexFactory.createIndex(this.VCFFile, codec, IndexType.TABIX);
                idx.write(new File(idxpath));
            } catch (final Exception e) {
                e.printStackTrace();
            }

        }

I bet it will work for GFF with suitable codec within htsjdk.