YOLOX is a high-performance anchor-free YOLO, exceeding yolov3~v5 with MegEngine, ONNX, TensorRT, ncnn, and OpenVINO supported. Documentation: https://yolox.readthedocs.io/
Apache License 2.0
9.15k
stars
2.15k
forks
source link
When running the Core ML model converted with coremltools in Swift, the objectness score behaves oddly only on the actual device #1780
I wanted to use YOLOX for iOS, so I created a yolox.mlpackage using coremltools and developed a Swift app. Although there were no issues in the simulator, when I tested it on an actual device (iPhone 15 Pro), I noticed that the objectness score was abnormally high. Here is the code for loading the model and using it in the app. When I run this code, the objectness score exceeds 1, although other values do not show any abnormalities. Could anyone provide some advice?
private func loadModel() {
do {
let model = try yolox()
self.model = try VNCoreMLModel(for: model.model)
} catch {
errorMessage = "Failed to load model: \(error.localizedDescription)"
print(errorMessage!)
}
}
private func classifyImage() {
print("classifyImage called")
guard let uiImage = image, let ciImage = CIImage(image: uiImage) else {
print("Image is not available for classification")
return
}
guard let validModel = model else {
print("Model is not loaded or available for classification")
return
}
let request = VNCoreMLRequest(model: validModel) { request, error in
if let error = error {
print("Classification failed with error: \(error.localizedDescription)")
return
}
guard let results = request.results as? [VNCoreMLFeatureValueObservation] else {
print("Classification returned no results or wrong results type.")
return
}
print("Total results returned: \(results.count)")
if results.isEmpty {
print("No results were returned.")
} else {
let boxes = self.postProcessFeatureValueObservations(results)
DispatchQueue.main.async {
self.displayResults(boxes: boxes, on: uiImage)
}
print(boxes)
}
}
let handler = VNImageRequestHandler(ciImage: ciImage)
do {
try handler.perform([request])
print("Classification request performed")
} catch {
print("Failed to perform classification: \(error.localizedDescription)")
}
}
func postProcessFeatureValueObservations(_ observations: [VNCoreMLFeatureValueObservation]) -> [BoundingBox] {
var boxes: [BoundingBox] = []
for observation in observations {
guard let multiArray = observation.featureValue.multiArrayValue else { continue }
let count = multiArray.count / 6
for i in 0..<count {
let index = i * 6
// セーフに各要素の値を取得
let centerX = CGFloat(multiArray[index].floatValue)
let centerY = CGFloat(multiArray[index + 1].floatValue)
let width = CGFloat(multiArray[index + 2].floatValue)
let height = CGFloat(multiArray[index + 3].floatValue)
let score = Float(multiArray[index + 4].floatValue)
let classScore = Float(multiArray[index + 5].floatValue)
let x1 = centerX - width / 2.0
let y1 = centerY - height / 2.0
let x2 = centerX + width / 2.0
let y2 = centerY + height / 2.0
let rect = CGRect(x: x1, y: y1, width: x2 - x1, height: y2 - y1)
if score * classScore > 0.1 {
print(score)
boxes.append(BoundingBox(classScore: classScore, score: score, rect: rect))
if score > 1{
print("over1")
}
}
}
}
I wanted to use YOLOX for iOS, so I created a yolox.mlpackage using coremltools and developed a Swift app. Although there were no issues in the simulator, when I tested it on an actual device (iPhone 15 Pro), I noticed that the objectness score was abnormally high. Here is the code for loading the model and using it in the app. When I run this code, the objectness score exceeds 1, although other values do not show any abnormalities. Could anyone provide some advice?