CNN's iReport is a citizen journalism initiative that allows people from around the globe to contribute pictures and video of breaking news stories from their own towns and neighborhood. According to Wikipedia, there were over 750,000 registered iReporters as of March 2011. That doesn't include other networks, blogs, community newspapers and so forth. This is a trend that's only going to grow over the coming years and it's a trend that Apple intends on tapping into. According to a new patent application that was recently published, Apple has invented two potential iPhone features simply referred to as "report" and "interview" modes. The new features creatively utilize multiplex video streaming techniques whereby dual cameras on the iPhone could automatically switch back and forth between interviewer and interviewee seamlessly for a live report. While Apple may be working on future iPhone features catering to iReporters, I think that their initial features will definitely capture the imaginations of iReporters everywhere.
An Overview of Apple's Invention
Apple's invention is focused on handheld wireless communication devices that have at least two cameras facing in opposite directions. The device receives a first video stream and a second video stream simultaneously from the two cameras. The device detects a speech activity of a person who is captured in the video streams by detecting the direction of sound or lip movement of the person. Based on the detection, the device automatically switches between the first video stream and the second video stream to generate a multiplexed video stream. The multiplexed video stream contains interleaving segments of the first video stream and the second video stream.
In one embodiment, the device detects the speech of a person by detecting the direction of sound. The device may include more than one microphone for detecting the direction of sound. A first microphone may point to the same direction to which one camera points, and a second microphone may point to the same direction to which the other camera points. Based on the direction of sound, the device automatically switches between the first video stream and the second video stream to generate a multiplexed video stream in which the video is, in a sense, "synchronized" to the speaker.
In another embodiment, the device detects the speech of a person by detecting the lip movement of the person. The device may include an image analyzer for analyzing the images in the video streams. Based on the lip movement, the device automatically switches between the first video stream and the second video stream to generate a multiplexed video stream.
In one embodiment, the multiplexed video stream may be transmitted to a far-end party by an uplink channel of a live video call. In another embodiment, the multiplexed video stream may be stored in the memory of the handheld communication device for viewing at a later time.
The handheld communication device may be configured or programmed by its user to support one or more of the above-described features.
Apple's Unique iReporter Mode
One of the key aspects of Apple's new patent application is squarely aimed at the modern day iReporter – Those in the public who capture a live event or incident anywhere, anytime, and then shares it with rest of the world via a website for greater distribution. It could be a citizen capturing a great photo of President Obama speaking at a rally or recording a coming tornado, live. Apple thinks that their latest invention will be able to assist iReporters that use an iPhone in many situations including live interviews.
What will it do and how will it Work?
Apple's proposed iPhone feature would be initiated by the user by either pressing a physical button positioned near the volume buttons or presented as a virtual button presented on the iPhone's UI as shown in our cover graphic. This particular feature that is generically referred to as the "auto-select" button allows the user to select the new "report mode." In Apple's patent FIG. 2 shown below we see a diagram illustrating a person using this proposed iPhone feature.
According to Apple, the user may hold this future iPhone in an upright position, such that the front-facing camera faces the user and the rear-facing camera faces a scene that the user wishes to capture in a video. The user may start a video capturing session that turns on both the front-facing camera and the rear-facing camera. The user may then activate the auto-select feature of the iPhone which would enable it to automatically switch between the two simultaneously captured video streams for a report.
Think of an iReporter filming a documentary on Elks in the Great White North. While he's filming the elk in the wild, the camera remains fixed on the elk and its natural surroundings. Then when the iReporter adds a narrative, for example, the iPhone would automatically switch camera views and focus on the iReporter speaking. When the iReporter falls silent, the camera would return to filming the elk. The two video streams may be interleaved throughout the multiplex video stream.
Apple states that the video could be transmitted live over various networks or recorded for later editing by the reporter at work or at home.
The iPhone's "Interview Mode"
In Apple's patent FIG. 3 below we see a diagram illustrating a scenario in which a first person may activate the auto-select feature of the iPhone to select "interview mode." The interview mode could be used when a person is recording a conversation with another person or when there are several people participating in a video conference.
In interview mode, a future iPhone may use both of the two microphones 113 and 114 to detect the direction of sound and to switch the video streams to face the direction of the detected sound. In another embodiment, the iPhone may use image processing techniques to detect lip movement of the first and second person.
Considering that the camera switches back and forth between subject and interviewer/reporter, I think that it's a very cool feature that will appeal to both the amateur and professional reporter alike.
Apple's patent FIG. 5 noted below is a block diagram illustrating an embodiment of the video processing module 480 which is the brains behind these new iPhone features involving multiplexed video streaming.
For those wishing to delve into the deeper side of the video processing unit which appears to control these new features, review patent points 032 through to 038. Apple's patent application 20110164105 was originally filed in Q1 2010 by inventors Jae Han Lee and E-Cheng Chang. It should be noted that although iReporter is a recognized CNN term, the general use of it in this report is referring to any Internet-Centric-Reporter. The only reference to iReporter in context with CNN is in our opening summery where it's properly identified as such.