MarBlog

Tag: Data portal

MEDIN – A way forward for UK Marine Science

by on Nov.06, 2011, under Data Management

The Marine Environmental Data & Information Network has been on the scene for quite a few years now.

I have been involved through my work, and attended the recent open partnership meeting, and it struck me that the recent move into an “operational phase” seems to have kickstarted a great leap forward. There were many more organisations present at the partnership meeting than I think has previously been there. Crucially, there is a mix of public bodies (including NDPB’s) and private companies now, and this is important as MEDIN is aiming to embed itself early into contractual agreements for the collection of Marine Data.

By embedding itself, I mean that MEDIN supplied a number of functions and standards that are useful for partners to include in data collection contracts. By including these, the contractor takes on an obligation to include metadata in relevant format, and to stick to standards for certain types of data.

The process for including and utilising MEDIN is that there are several Data Archive Centres (DAC’s) which store data long term and make it available, initially through MEDIN’s discovery portal, but also with view and download services planned for many areas. While there may be costs associated with committing large data sets to an archive centre, there are actually a number of great benefits, which I think are not fully realised by partners or data suppliers:

  • Data are stored securely and handled by experts.
  • Data are made public (if so agreed)
  • The partner does not incur the costs of storing, backing up and maintaining Master Data Management aspects of these data sets
  • Partners bandwidth costs are not escalated by starting to publish data.
  • Data becomes part of a UK wide network of reporting data
Each of the Data Archive Centres are accredited, and as such are secure places to put your data in terms of continued funding, secure handling and storage (it’s what they do for a living in most cases).
At the partnership meeting in London, there was a good debate about both partners who had great success with committing data to MEDIN, but also an open and frank discussion about a general reluctance towards submitting data sets. Some of the reasons discussed was that adding the MEDIN criteria to contracts would push up cost or limit the amount of data that could be collected at same cost. Further, submitting already held data was generally something people wnated to do, but found to have very little time towards, and little or no chance for spending money on getting the data curated with an external body (in this case a Data Archive Centre).
Such concerns are fully understandable, but is also a cause for concern. For one, it quite clearly demonstrates that many organisations  have difficulty in putting a maintenance costs on their data sets,  and that perhaps there sometimes is a sacrifice of quality over quantity of data.  Secondly, there are a number of costs not yet accounted for in holding your own data. In particular Marine Environmental data that will fall under the European INSPIRE Directive will be required to be made available through Discovery, View, Download and Transformation services. If each partner or data provider is to establish, run and maintain such services, it will be significantly more expensive than using the service offered by DAC’s.
So while MEDIN is certainly gathering pace, and now published more than 2,000 metadata records on their portal, with more coming in, and more work planned, there are certainly also challenges ahead. Not the least will be to create a more transparent process (and possibly even cost) for data providers to commit their (often multidisciplinary) data into (often discipline specific) DAC’s.
The time ahead will be exciting. There are clear indication that more organisations are sitting up and paying attention to the work done by MEDIN, and I think the next couple of years will be crucial to determine if the processes and standards can be made to work in a way that is streamlined enough to really encourage partners to get data up to those Archive Centres and to get them published so all will benefit.
Leave a Comment :, , , , , , , , more...

BODC’s New data catalogue

by on Nov.24, 2010, under Online Data Sources

British Oceanographic Data Centre has jsut announced a new facility on their website to search and retrieve data series directly from the web. While a lot of data could be retrieved before, this catalogue truly opens up access across all categories and project, with over 76,000 data series being put online in a searchable format. The series are mainly CTD casts, but also include bathymetry meterology, optical properties, wave data and more.

The great thing is that data is available in several recognised formats, NetCDF, ODV and ASCII files – so virtually everyone in the field can access this data in a preferred format.

There are some limitations in terms of the way you can refine searches, but most of them makes sense from the perspective of optimising searches and not hanging up the server in searches that return virtually everything.

By the time you have narrowed your search criteria to return 1,000 series or less, you can retrieve results. There’s the option of downloading a KML file of coverage, and you can retrieve data in your preferred format.

It is important to note that we’re talking data series, not individual points here, so even a single series can contain thousands of data points, giving you access to a seriously large amount of oceanographic data with a wide geographic coverage.

The initial map on the start page show waters around Britain, but make sure you either zoom out or pan around as there is data from a much wider region – virtually all of thw world - than what is shown on the map.

You do have to register an account with BODC in order to checkout your “data shopping”, but there is a huge amount of data freely available. The map tells you up front which data series are freely available.

BODC has truly made their data a lot more accessible with this exercise.

1 Comment :, , , , more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Blogroll

A few highly recommended websites...