基于云平台的海量高光谱数据管理系统设计与实现

实现了基于云平台的海量高光谱数据管理系统。系统提供高光谱图像上传、图像格式转换、以及光谱特征分析的功能。


摘要:高光谱遥感图像数据具有很高的光谱分辨率,可以在电磁波谱的可见光、近/中/热红外等波段范围内获取许多很窄的光谱波段信息,从而得到光谱连续的影像数据,这些数据已经成为农业遥感、国土资源利用、大气研究、环境监测、地质勘探等领域的重要手段,应用极其广泛。但由于高光谱图像具有波段多且光谱分辨率高的特点,使得高光谱遥感数据的处理算法具有复杂度高、硬件资源需求大等问题,如何合理的存储海量的高光谱遥感数据并管理成为一个亟待解决的问题。云计算技术的出现,为高光谱图像的存储带来了新的解决方案,云计算的分布式计算能力有效地解决了高光谱遥感处理算法的单机瓶颈问题,其可扩展的存储能力为海量的高光谱数据提供了更广阔的平台。

本文在分析当前的高光谱遥感图像的数据存储与管理研究现状的基础,基于HDFS文件系统和MapReduce编程模型设计了一个高光谱遥感图像的数据管理系统,其中主要包括了高光谱遥感图像上传、高光谱遥感元数据管理、原始图像数据存储管理设计、高光谱遥感图像格式转换等核心模块,给出了系统具体详细设计方案,在基于SpringMVC框架和HDFS分布式文件系统进行了系统开发和实现,同时结合光谱角匹配算法、光谱信息散度匹配算法和光谱相似匹配算法,实现了高光谱遥感图像的光谱特征分析。基于云平台实现海量数据的存储管理系统可以充分利用云计算的灵活性,为存储和管理海量高分辨率遥感影像数据提供了一种有效的解决方案,有助于提高使用者的处理效率,同时使用云平台可以按需使用的特性,使社会资源得到更合理的分配使用。

关键词  高光谱遥感图像  海量数据管理  云计算  Hadoop  光谱特征 HDFS

Title   Distributed Parallel Optimization Method of Hyperspectral Images Mixed Pixel Unmixing    

Abstract:Hyperspectral remote sensing image data has a high spectral resolution, can be in the electromagnetic spectrum of visible light, near / mid / hot infrared band range to obtain many very narrow spectral band information, resulting in spectral continuous image data, these data have become an important method in agricultural remote sensing, land use, atmospheric research, environmental monitoring and geological prospecting, and is widely used. However, because the hyperspectral image has many characteristics of high band and high spectral resolution, the processing algorithm of hyperspectral remote sensing data has the problems of high complexity and large demand of hardware resources. How to store large amount of hyperspectral remote sensing data and manage it as a reasonable problems to be solved. The emergence of cloud computing technology brings a new solution for the storage of hyperspectral images. The distributed computing power of cloud computing effectively solves the single bottleneck problem of hyperspectral remote sensing algorithm, and its scalable storage capacity is massive Hyperspectral data provides a broader platform.

Based on the analysis of the current situation of data storage and management of hyperspectral remote sensing images, a data management system based on HDFS file system and MapReduce programming model is designed, which mainly includes hyperspectral remote sensing image uploading, high Spectrum remote sensing metadata management, original image data storage management design, hyperspectral remote sensing image format conversion and other core modules, given the specific detailed design of the system, based on SpringMVC framework and HDFS distributed file system for system development and implementation, while The spectral feature analysis of hyperspectral remote sensing image is realized by combining spectral matching algorithm, spectral information pergence matching algorithm and spectral similarity matching algorithm. The storage management system based on cloud data can make full use of the flexibility of cloud computing and provide an effective solution for storing and managing massive high-resolution remote sensing image data, which can improve the processing efficiency of users. The use of cloud platform can be used on the characteristics of the use of social resources to be more rational use of distribution.