Visual Model Encoders

[BaiYunpeng1949]

The heads-up-computing model is an extensive entity from the user-in-the-box community, it serves as a simulation to the realistic human eyes and relevant visual perception abilities.

1. Monocular FOV: The monocular visual field consists of central vision, which includes the inner 30 degrees of vision and central fixation, and the peripheral visual field, which extends 100 degrees laterally, 60 degrees medially, 60 degrees upward, and 75 degrees downward. A normal visual field is an island of vision measuring 90 degrees temporally to central Fixation, 50 degrees superiorly and nasally, and 60 degrees inferiorly. Visual acuity increases from movement discrimination in the extreme peripheral vision to better than 20/20 in the center of vision. Depression or absence of vision anywhere in the island of vision is abnormal. [Ref]

2. Models of eye movements, visual encodings, and interaction with information processing, ref from: An integrated model of eye movements and visual encoding [Ref].

3. To be added.

[BaiYunpeng1949]

Integrating with mathematical models examples.

1. Env configurations: 1) Preset monocular fov as 90 degrees to simulate human's fov.
2. Action space: 1) Add noises to the eyeball rotations to simulate saccades. 2) Add addtional heads motions to adjust the smart glasses content gap position to see clearer. The comfortableness could be added as a bounding factor - if the agent is always tilting his head trying to see things, he will become uncomfortable and little penalties should be applied.
3. Observation space: 1) Mask/Change the resolution/Distore the peripheral rgb pixels to simulate human's central vision/foveated vision/fisheye vision.
4. To be added.

[BaiYunpeng1949]

As for the foveal vision, we can use a Guassian filter to blur the peripheral vision.

The foveal vision angle could be 1-2 degrees (the most acuitive part) to 30-60 degrees, less acuitive peripheral parts.