Deep learning reconstruction enables full-Stokes single compression in polarized hyperspectral imaging

Due to the rich information reflected, polarized hyperspectral imaging has been widely applied in environmental monitoring, biological diagnosis, food safety, and other fields. In terms of technology development, polarized imaging is mainly based on Fourier transform, pixelated polarizers, and compressive sensing (CS). Currently, all the above three methods can achieve full-Stokes polarized imaging. Typically, Fourier transform imaging spectropolarimetry based on polarization modulation array (PMAFTISP) requires only one acquisition to obtain full-Stokes images. The PMAFTISP includes three polarization modulation arrays and three independent optical elements. System complexity and channel crosstalk may affect imaging quality. In addition, pixelated full-Stokes polarimeters require rotating polarizers or designing metasurfaces. Moreover, the fabrication of precision pixelated devices is costly and time-consuming. Recently, compressive full-Stokes polarimeters are constructed with only two commercial components, providing an easy-to-operate and time-saving system. Full-Stokes images can be reconstructed from two measurements compressed by a quarter-wave plate (QWP) and a liquid crystal tunable filter (LCTF). Furthermore, benefiting from a retarder followed by a Wollaston prism with splitting effect, full-Stokes images can be reconstructed from one measurement. Nevertheless, the above compressive polarimeters all rely on traditional reconstruction methods, such as two-step iterative shrinkage/threshold (TwIST) algorithm, which require careful selection of polarization parameters and sparse basis.

The research team of Professor Xu Tingfa from Beijing University of Technology has developed full-Stokes single compression in polarized hyperspectral imaging by introducing deep learning reconstruction (DL-FSCPHI). Full-Stokes images of the target light are compressed into one measurement by a QWP combined with an LCTF. The full Stokes images are then reconstructed simultaneously by a convolutional neural network (CNN). The relevant results were published in Volume 21, Issue 5 of Chinese Optics Letters (Axin Fan, et al. Deep learning reconstruction enables full-Stokes single compression in polarized hyperspectral imaging) and was selected as the cover paper.

The shown figure 1 illustrates the overall schematic diagram of DL-FSCPHI method comprised of imaging system and polarization reconstruction. The imaging system mainly consists of a light source (Thorlabs, OSL2), a QWP (Thorlabs, SAQWP05M-700), an LCTF (Thorlabs, KURIOS-VB1/M), and a complementary metal oxide semiconductor (CMOS) detector (Basler, acA2040-180km). The polarization state of the target light is expressed by a column vector composed of four Stokes parameters. The polarization characteristics of both QWP and LCTF are described by a Mueller matrix with 16 elements in four rows and four columns. Furthermore, LCTF and QWP Mueller matrices are multiplied to obtain the system Mueller matrix. Then, the system Mueller matrix is multiplied by the Stokes column vector of the target light to obtain the Stokes column vector of the modulated light. The modulated first Stokes parameter representing the total light intensity is finally measured by the CMOS detector. Herein, the QWP and LCTF polarization angles are fixed and the LCTF center wavelength is switched to obtain the polarized hyperspectral images of the target under full-Stokes single compression.

The polarization reconstruction involves model training and model testing. Initially, a CNN model composed of two convolution layers is built on Keras framework. The first convolution layer expands one compressed image into multiple images, and the second convolution layer enhances the image details to predict full-Stokes images. The model is then trained utilizing full-Stokes images and compressed images of 60 targets with 400×400 spatial pixels in 18 spectral bands. The model is tested utilizing full-Stokes images and compressed images of other 7 targets with 400×400 spatial pixels in 18 spectral bands. In order to fully verify the reliability of DL-FSCPHI method, two polarization angles, two model structures, and two training parameters are designed. Compared with typical TwIST algorithm, the average peak signal to noise ratio (PSNR) and structural similarity (SSIM) values are improved by 13.55 dB and 0.28, respectively.

This work demonstrates the great promise to develop deep learning reconstruction for full-Stokes single compression and other applications. In the future, it is worth investigating a stronger universal model that is widely applicable to different imaging systems and different polarization angles. Meanwhile, deep learning reconstruction is being extended to four-dimensional compressive imaging, including one-dimensional polarized, one-dimensional spectral, and two-dimensional spatial compression.

深度学习下的全偏振高光谱计算成像

偏振高光谱成像可以反映丰富的信息，在环境监测、生物诊断、食品安全等领域得到广泛应用。偏振成像主要基于傅里叶变换、像素化偏振器和压缩感知。目前，以上三种方法均可实现全斯托克斯偏振成像。其中，基于偏振调制阵列的傅里叶变换成像光谱偏振法（PMAFTISP）仅需一次采集即可获得全斯托克斯图像，PMAFTISP包括三个偏振调制阵列和三个独立的光学元件，系统复杂和信道串扰将影响成像质量。此外，像素化的全斯托克斯偏振计需要旋转偏振器或设计超表面，精密像素化器件的制造成本高、耗时长。而压缩全斯托克斯偏振仪仅由两个商用组件构成，提供了易于操作且节约时间的系统，通过四分之一波片（QWP）和液晶可调滤波器（LCTF）进行压缩，可以从两次测量中重建全斯托克斯图像。同时，利用延迟器后接具有分束作用的Wollaston棱镜，可以从一次测量中重建全斯托克斯图像。然而，上述压缩偏振仪都依赖于传统的重建方法，如两步迭代收缩/阈值（TwIST）算法，需要谨慎选择偏振参数和稀疏基。

北京理工大学许廷发教授研究团队通过引入深度学习重建，开发了全斯托克斯单次压缩的偏振高光谱成像（DL-FSCPHI）系统。目标光的全斯托克斯图像由QWP与LCTF联合压缩到一次测量中，然后通过卷积神经网络（CNN）同时重建完整的斯托克斯图像。相关成果发表在Chinese Optics Letters第21卷第5期上(Axin Fan, et al. Deep learning reconstruction enables full-Stokes single compression in polarized hyperspectral imaging)，并被选为当期封面。

DL-FSCPHI方法的总体原理图如图1所示，包括成像系统和偏振重建。成像系统主要由光源、QWP、LCTF和互补金属氧化物半导体（CMOS）探测器组成。目标光的偏振状态由四个斯托克斯参量组成的列向量表示，QWP和LCTF的偏振特性均用一个四行四列含16个元素的穆勒矩阵来描述，LCTF与QWP的穆勒矩阵相乘得到系统的穆勒矩阵。然后，将系统的穆勒矩阵与目标光的斯托克斯列向量相乘，得到调制光的斯托克斯列向量。调制后的第一斯托克斯参量代表总光强，最后由CMOS探测器测量。在此，固定QWP和LCTF的偏振角度，切换LCTF的中心波长，得到全斯托克斯单次压缩下目标的偏振高光谱图像。

偏振重建包括模型训练和模型测试。首先，在Keras框架上构建由两个卷积层组成的CNN模型。第一层卷积层将一幅压缩图像扩展成多幅图像，第二层卷积层增强图像细节以预测全斯托克斯图像。然后，利用60个目标在18个光谱波段下具有400×400个空间像素的全斯托克斯图像和压缩图像训练模型，利用另外7个目标在18个光谱波段下同样具有400×400个空间像素的全斯托克斯图像和压缩图像测试模型。为了充分验证DL-FSCPHI方法的可靠性，设计了两个偏振角度、两种模型结构和两组训练参数。与典型TwIST算法相比，平均峰值信噪比（PSNR）和结构相似度（SSIM）值分别提高了13.55 dB和0.28。

该项工作展示了全斯托克斯单次压缩和其他应用开发深度学习重建的巨大前景。未来，有必要研究一种更强大的通用模型，以广泛适用于不同的成像系统和不同的偏振角度。同时，该团队正在研究将深度学习重建扩展到四维压缩成像，包括一维偏振、一维光谱和二维空间压缩。