We are inundated with environmental data – Earth observing satellites stream terabytes of data back to us daily; ground-based sensor networks track weather, water quality, and air pollution, taking readings every few minutes; and community scientists log hundreds and thousands of observations every day, recording everything from bird sightings to road closures and accidents. But this very richness of data has created a new set of problems.
This four-part post gives a high-level view of some of the challenges of big data in the geosciences – and how they might be solved – loosely based on the Earth and Space Science Informatics sessions and town halls at the AGU fall meeting in Dec 2016.
With so much environmental data, looking for a specific dataset for a research project can sometimes feel like looking for a needle in a haystack. How can data discovery, that is, finding the right dataset or sharing one’s own dataset with the larger research community, be made more efficient?