The Future
The field of super slow-motion video is an active research field and is constantly changing and getting new breakthroughs from research institutes such as MIT and Caltech. New cameras and algorithms are getting introduced and new systems are now capable of filming at over a trillion frames per second. These incredible speeds are unlocking the ability to see physical phenomena such as the speed of light to now be seen on video. One such system that has been developed is called FRAME. FRAME (Frequency Recognition Algorithm for Multiple Exposures), introduced in 2017, is an improvement on previous femtosecond streak cameras. Able to capture up to a trillion frames per second, the FRAME system is an advanced filming technique, which unlike other techniques before it, has the capability to perform ultrafast spectroscopic videography of dynamic single events. Another technique, introduced in 2018, is T-CUP (Trillion-frame-per-second Compressed Ultrafast Photography), which can film up to 10 trillion frames per second. Both FRAME and T-CUP are improvements on previous systems of ultra slow motion video capture, which operated with standard streak cameras. Of course, these systems are extremely complex, and most of their details are omitted in this section for the sake of providing a digestible overview. To learn more about how exactly they function, see the FRAME and T-CUP papers cited in the references section.
Streak Camera
Streak cameras are used to measure the pulse duration of some ultrafast laser systems and for applications such as time-resolved spectroscopy and LIDAR. They operate by transforming the time variations of a light pulse into a spatial profile on a detector, by causing a time-varying deflection of the light across the width of the detector. A light pulse enters the instrument through a narrow slit along one direction, and gets deflected perpendicularly, so that photons that arrive first hit the detector at a different position compared to photons that arrive later. This forms a “streak” of light on the resulting image, from which the duration and other temporal properties of the light pulse can be inferred [12].
Figure 1: Operation of the Streak Camera [13]
FRAME
“Many important scientific questions in physics, chemistry and biology require effective methodologies to spectroscopically probe ultrafast intra- and inter-atomic/molecular dynamics. However, current methods that extend into the femtosecond regime are capable of only point measurements or single-snapshot visualizations and thus lack the capability to perform ultrafast spectroscopic videography of dynamic single events. Here we present a laser-probe-based method that enables two-dimensional videography at ultrafast timescales (femtosecond and shorter) of single, non-repetitive events.”
The novelty of the FRAME concept lies in superimposing a structural code onto the illumination to encrypt a single event, which is then deciphered in the data post processing. Because each image in the video sequence is extracted by using a unique spatial code, the method does not rely on a specific optical wavelength or laser bandwidth, and hence can be used for spectroscopic measurements [14].
Figure 2: FRAME’s Operating Principle [14]
In Figure 2 above, (a) shows a uniformly illuminated sample with an image that resides mostly near the origin in the Fourier domain. The outer circle marks the resolution limit of the detector. (b) Shows the sample being illuminated with sinusoidal intensity modulation, effectively placing two ‘image copies’ of the object structures in the otherwise unexploited space in the Fourier domain. In (c, d), each ‘image copy’ fills only a fraction of the available reciprocal space, thereby allowing for multiple-illumination schemes without signal cross-talk [14].
T-CUP
T-CUP (Trillion-frame-per-second Compressed Ultrafast Photography), introduced in 2018, is a method of capturing up to 10 trillion frames per second. Previously established ultrafast imaging techniques either struggle to reach the desired exposure time or require repeatable measurements. In the paper, the T-CUP technique is asserted to be better than the FRAME technique; “such approaches are incapable of imaging luminescent transient events, so they are precluded from direct imaging of evolving light patterns in temporal focusing. The requirement of active illumination was recently eliminated by CUP, a new single-shot ultrafast imaging modality. Synergizing compressed sensing and streak imaging, CUP works by first compressively recording a three dimensional scene into a two-dimensional snapshot and then computationally recovering it by solving an optimization problem” [15].
"We knew that by using only a femtosecond streak camera, the image quality would be limited," says professor Lihong Wang, the Bren Professor of Medical Engineering and Electrical Engineering at Caltech and the Director of Caltech Optical Imaging Laboratory (COIL). "So to improve this, we added another camera that acquires a static image. Combined with the image acquired by the femtosecond streak camera, we can use what is called a Radon transformation to obtain high-quality images while recording ten trillion frames per second" [16].
The operation of T-CUP consists of data acquisition and image reconstruction. For the data acquisition, the intensity distribution of a 3D spatiotemporal scene, I[m,n,k], is first imaged with a beam splitter to form two images. The first image is directly recorded by a 2D imaging sensor via spatiotemporal integration (defined as spatial integration over each pixel and temporal integration over the entire exposure time). This process, which forms a time-unsheared view with an optical energy distribution of Eu[m,n], can be expressed by
𝐸u[𝑚,𝑛] = 𝜂∑𝑘(ℎu∗𝐼)[𝑚,𝑛,𝑘]
where η is a constant, hu represents spatial low-pass filtering imposed by optics in the time-unsheared view, and * denotes the discrete 2D spatial convolution operation [15].
Figure 3: Schematic of the T-CUP System [15].
By improving the frame rate by two orders of magnitude compared with the previous state-of-the-art, T-CUP demonstrated that the ever-lasting pursuit of a higher frame rate is far from ending. As the only detection solution so far available for passively probing dynamic self-luminescent events at femtosecond timescales in real time, T-CUP was used to reveal spatiotemporal details of transient scattering events that were inaccessible using previous systems. The compressed-sensing-augmented projection extended the application of the Radon transformation to probing spatiotemporal datacubes. This general scheme can be potentially implemented in other imaging modalities, such as tomographic phase microscopy and time-of-flight volumography. T-CUP’s unprecedented ability for real-time, wide-field, femtosecond-level imaging from the visible to the near-infrared will pave the way for future microscopic investigations of time-dependent optical and electronic properties of novel materials under transient out-of-equilibrium conditions. With continuous improvement in streak camera technologies, future development may enable a 1 quadrillion fps (10^15 fps) frame rate with a wider imaging spectral range, allowing direct visualization and exploration of irreversible chemical reactions and nanostructure dynamics [15].
