I have an idea for how to process the raw image:
- Use the calibration information in each photo to make a model of the lenses
- Create a normal map of the raw image show where each ray came from
- Create a color map of the raw image for future demosaicing
- Project rays from the modeled CCD and ray trace it to your selected focal plane
- Apply some filter to demosaic and rasterize
I still need to process for a depth map. Maybe look for where rays converge?