App Crashes on vosk.loadMedel Dynamically (Expermintal)

[BiskremMuhammad]

Hi, I couldn't make the pods install successfully inside my ios folder, although i made all steps correctly like the README. but anyway, i tried to use the expermintal method and load the model dynamically. the app loads up in loading state until the model is downloaded, but immediately crashes when passing the downloaded folder to vosk.loadModel, here is a full code

import * as FileSystem from "expo-file-system";
import JSZip from "jszip";

// Model URL and directory where the model will be stored
const modelUrl =
  "https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip"; // Replace with your model's URL
const modelDir = `${FileSystem.documentDirectory}vosk-model`;

export const downloadAndUnzipModel = async () => {
  try {
    // Check if model already exists
    let modelPath;
    const modelExists = await FileSystem.getInfoAsync(modelDir);
    if (!modelExists.exists) {
      // Define zip path where the model will be downloaded
      const zipPath = `${FileSystem.documentDirectory}vosk-model.zip`;

      // Download the model zip file
      const downloadResumable = FileSystem.createDownloadResumable(
        modelUrl,
        zipPath
      );

      const downloadResult = await downloadResumable.downloadAsync();
      if (downloadResult?.uri) {
        console.log("Model downloaded successfully:", downloadResult.uri);

        // Read the zip file and extract it
        const zipData = await FileSystem.readAsStringAsync(downloadResult.uri, {
          encoding: FileSystem.EncodingType.Base64,
        });

        const zip = new JSZip();
        const content = await zip.loadAsync(zipData, { base64: true });

        // Create the directory for the unzipped model
        await FileSystem.makeDirectoryAsync(modelDir, { intermediates: true });

        // Unzip each file and save it to the model directory
        for (const [filename, file] of Object.entries(content.files)) {
          if (!file.dir) {
            const fileData = await file.async("uint8array");
            const filePath = `${modelDir}/${filename}`;
            await FileSystem.writeAsStringAsync(
              filePath,
              Buffer.from(fileData).toString("base64"),
              {
                encoding: FileSystem.EncodingType.Base64,
              }
            );
            console.log(`Extracted: ${filePath}`);
          }
        }
        console.log("Model downloaded and unzipped successfully");
      } else {
        console.error("Failed to download the model");
      }
    } else {
      console.log("Model already exists, skipping download", modelDir);

      // List contents of modelDir
      const contents = await FileSystem.readDirectoryAsync(modelDir);
      modelPath = contents[0];
      if (contents.length > 0) {
        const firstSubdirectory = `${modelDir}/${contents[0]}`;
        try {
          const subdirectoryContents = await FileSystem.readDirectoryAsync(
            firstSubdirectory
          );
          console.log(
            `Contents of ${contents[0]} directory:`,
            subdirectoryContents
          );
        } catch (error) {
          console.error(
            `Error reading contents of ${contents[0]} directory:`,
            error
          );
        }
      } else {
        console.log("No subdirectories found in modelDir");
      }
    }

    return modelPath || undefined;
  } catch (error) {
    console.error("Error downloading or initializing the model:", error);
  }
};

and inside my screen i did this:

import { Button, Image, StyleSheet } from "react-native";

import { HelloWave } from "@/components/HelloWave";
import ParallaxScrollView from "@/components/ParallaxScrollView";
import { ThemedText } from "@/components/ThemedText";
import { ThemedView } from "@/components/ThemedView";
import { useState, useEffect, useRef } from "react";
import { downloadAndUnzipModel } from "@/src/utils/downloadModel";
import Vosk from "react-native-vosk";

export default function HomeScreen() {
  const [loading, setLoading] = useState<boolean>(true);
  const vosk = useRef<Vosk | null>(null);
  const [result, setResult] = useState<string>("");
  const [recognizing, setRecognizing] = useState<boolean>(false);

  useEffect(() => {
    const downloadModelAsync = async () => {
      const modelDir = await downloadAndUnzipModel();
      console.log("Model is downloaded and unzipped");
      try {
        if (modelDir) {
          vosk.current = new Vosk();
          await vosk.current.loadModel(modelDir);
          vosk.current
            ?.start()
            .then(() => {
              console.log("Starting recognition...");
              setRecognizing(true);
            })
            .catch((e) => console.error("Error starting recognition:", e));
        }
      } catch (error) {
        console.error("Error loading model:", error);
      }
      setLoading(false);
    };
    downloadModelAsync();
  }, []);

  useEffect(() => {
    if (!vosk.current) return;
    const resultEvent = vosk.current.onResult((res) => {
      console.log("An onResult event has been caught: " + res);
      setResult(res);
    });

    const partialResultEvent = vosk.current.onPartialResult((res) => {
      setResult(res);
    });

    const finalResultEvent = vosk.current.onFinalResult((res) => {
      setResult(res);
    });

    const errorEvent = vosk.current.onError((e) => {
      console.error(e);
    });

    const timeoutEvent = vosk.current.onTimeout(() => {
      console.log("Recognizer timed out");
      setRecognizing(false);
    });

    return () => {
      resultEvent.remove();
      partialResultEvent.remove();
      finalResultEvent.remove();
      errorEvent.remove();
      timeoutEvent.remove();
    };
  }, [vosk]);

  return (
    <ParallaxScrollView
      headerBackgroundColor={{ light: "#A1CEDC", dark: "#1D3D47" }}
      headerImage={
        <Image
          source={require("@/assets/images/partial-react-logo.png")}
          style={styles.reactLogo}
        />
      }
    >
      <ThemedView style={styles.titleContainer}>
        <ThemedText type="title">Welcome!</ThemedText>
        <HelloWave />
      </ThemedView>
      <ThemedView style={styles.stepContainer}>
        <ThemedText type="subtitle">
          {loading ? "Loading..." : "Ready, start recording"}
        </ThemedText>
      </ThemedView>
      <ThemedView style={styles.stepContainer}>
        <ThemedText type="subtitle">
          {result ? `Result: ${result}` : "No result yet"}
        </ThemedText>
      </ThemedView>
      <ThemedView>
        {!loading && !recognizing && (
          <Button
            onPress={async () => {
              if (vosk.current) {
                await vosk.current.start();
                setRecognizing(true);
              }
            }}
            title="Start Recording"
          />
        )}
        {!loading && recognizing && (
          <Button
            onPress={() => {
              if (vosk.current) {
                vosk.current.stop();
                setRecognizing(false);
              }
            }}
            title="Stop Recording"
          />
        )}
      </ThemedView>
    </ParallaxScrollView>
  );
}

const styles = StyleSheet.create({
  titleContainer: {
    flexDirection: "row",
    alignItems: "center",
    gap: 8,
  },
  stepContainer: {
    gap: 8,
    marginBottom: 8,
  },
  reactLogo: {
    height: 178,
    width: 290,
    bottom: 0,
    left: 0,
    position: "absolute",
  },
});

so I added the permissions to Info.plist

<key>NSMicrophoneUsageDescription</key>
<string>This app requires access to the microphone for speech recognition.</string>

so the downloaded folder is correct and here is the contents from the downloaded folder as shown in the logs

 LOG  Contents of modelDir:
 LOG  Contents of vosk-model-small-en-us-0.15 directory: ["README", "am", "graph", "ivector", "conf"]
 LOG  Model is downloaded and unzipped

and then the app crashes with this error:

ERROR (VoskAPI:Model():model.cc:122) Folder '/private/var/containers/Bundle/Application/479BB9D6-8897-485E-A2E8-A950D1A64F6B/demo.app/vosk-model-small-en-us-0.15' does not contain model files. Make sure you specified the model path properly in Model constructor. If you are not sure about relative path, use absolute path specification.

[joaolobao380]

@BiskremMuhammad I have the same problem, However, I'm not using dynamic linking, and it only happens on iOS, have you come up with a solution?

[BiskremMuhammad]

@joaolobao380 It never worked with me with native module for ios, (download the model and added the folder to xcode), but i manage to solve it and load the model successfully with dynamic load, using a download utility and let the app download the model internally and unzip it. so my solution included the above code, but at the end i removed the file:// from the downloaded model path and it worked.

[joaolobao380]

perfect!! Thank you!

[ShristiC]

This is what should be happening with model loading- try to load from a dynamic path, otherwise default to the app bundle. If the static loading is not working, there may be some helpful logs when running the app from XCode. What are the logs you are seeing with the static load?

[riderodd]

Can you follow the updated README and let us know if it still crashes ?

[riderodd]

I can confirm that with a trailing file:// it does not work

[ShristiC]

Created this PR, please let me know if it resolves your issue! https://github.com/riderodd/react-native-vosk/pull/79