k2-fsa / k2

FSA/FST algorithms, differentiable, with PyTorch compatibility.
https://k2-fsa.github.io/k2
Apache License 2.0
1.08k stars 211 forks source link

Fix fsa_from_str #1287

Closed pkufool closed 2 months ago

pkufool commented 2 months ago

Fix https://github.com/k2-fsa/k2/issues/1286

I think there should be something wrong with reading float from istringstream, here are some tests:

#include <iostream>
#include <sstream>
#include <string>
#include <iomanip>

int main(){
  std::string str = "40000000 40000111 40000110 40000102 2147483647";
  std::istringstream iss(str);
  float aux;
  while (iss >> aux) {
    if ((int32_t)aux != aux) {
      std::cout << "require int" << std::endl;
    }
    std::cout << "float : " << std::setprecision (15) << aux << " int32 : " << (int32_t)aux << std::endl;
  }
}

The output is:

float : 40000000 int32 : 40000000
float : 40000112 int32 : 40000112
float : 40000112 int32 : 40000112
float : 40000104 int32 : 40000104
float : 2147483648 int32 : 2147483647

When changing to double :

#include <iostream>
#include <sstream>
#include <string>
#include <iomanip>

int main(){
  std::string str = "40000000 40000111 40000110 40000102 2147483647";
  std::istringstream iss(str);
  double aux;
  while (iss >> aux) {
    if ((int32_t)aux != aux) {
      std::cout << "require int" << std::endl;
    }
    std::cout << "float : " << std::setprecision (15) << aux << " int32 : " << (int32_t)aux << std::endl;
  }
}

The output is:

float : 40000000 int32 : 40000000
float : 40000111 int32 : 40000111
float : 40000110 int32 : 40000110
float : 40000102 int32 : 40000102
float : 2147483647 int32 : 2147483647

I have not looked into the source code of standard library to figure out the deeper reasons, this is just a fix of the issue in k2.