A monumental challenge for engineers is not just how to acquire data in testing, but how to store and preserve such large volumes of it
The electric vehicle testing community is having to deal with increasingly complex products and faster development cycles, meaning that test labs are under constant pressure. As such, the ability to efficiently analyze test data has never been more critical.
Costs must be kept as low as possible, while growing amounts of data from sensors are processed. Engineers must monitor and analyze test data in real time, regardless of how much there is to sift through. It’s therefore no wonder that they are seeking new ways to improve testing efficiency and reduce risk of low quality data.
During acquisition, electrical and mechanical signals are measured at sample rates in the kilohertz range and need to be highly synchronized. These tests in particular generate an overwhelming amount of data. The real challenge is storing and preserving this data.
The handling of such large volumes of structured and unstructured data requires an adaptive and scalable data back end. To cope with ever-changing requirements, setup configurations, parameter extensions and sample rates, a separation between hot and cold datais helpful.
‘Cold data’, which is less frequently accessed and only needed for auditing or post-processing, is stored in a distributed streaming platform that scales as needed.
The ‘hot data’, which is accessed in real time for analysis, is provided in a NoSQL time-series database. This database stores readings securely in redundant, fault-tolerant clusters. Flexible data aggregation ensures that the right granularity is available at any time. Data can then be replayed and aggregated in different ways for detailed analysis of test events. This approach minimizes IT operational costs and the storage infrastructure within the test lab.