March 13, 2019
Weather forecasting for landscapes
Whether or not to head out at the weekend is always a challenge – which is the best weather service?
The last weekend was due to be my last Seascape & Landscape workshop of the spring season. The aim of the workshop is to give attendees lots of opportunities to use different types of filters. Unfortunately, this means that if we are expecting lots of rain, that it is best to postpone the session. That is what we had to do this week. However, it gave me the opportunity to look at different weather services and see how accurate their forecast was. I usually start to check weather forecasting sites three days before the workshop. I use seven different weather services to try and get a wide opinion of what the weather will be like. On this occasion, I decided that I would retain a record of each of the forecasters to determine in a reasonably scientific way, which was the best. I’ll add a bit of a health warning – this isn’t a definitive view of which will be the best in the future. It’s based on a fairly limited set of data. It does, though, give an indication of how well they performed.
Weather forecasting services
The seven services I use are Darksky.Net, BBC Weather, Met Office, Weather Underground, Accu Weather, Yr and Metcheck Weather. They all use different methodologies of trying to predict the weather.
Dark Sky uses a crowd-sourced model where they collect information from lots of small weather stations. They then use this data with their own robots to process what they claim to be a hyper-local weather forecast. The company was formed as a Kickstarter in 2011.
BBC Weather is provided by the Meteo Group. It’s the weather of the UK’s public service broadcaster. Meteo Group was formed in 1986 and claims to use accurate science with advanced technology and local expertise.
The Met Office is the UK’s National weather service. It is funded by the UK Government and uses weather satellites as well as land-based weather stations. The Met Office was established in 1854.
Weather Underground was formed in 1993. Similar to Dark Sky, it gathers information from over 250,000 members who send real-time data from their personal weather stations. They then combine this weather of local data with in-house meteorologists to produce a localised forecast.
AccuWeather was formed in 1962. They have over 100 meteorologists and are US based. Most of their data come from the National Weather Service. In 2018 they were judged to be the most accurate 1-5 day forecasting service for wind, rain and high temperature.
Yr was launched in 2007. It’s a collaboration between the Norwegian Meteorological Institute and the Norwegian Broadcasting Corporation. They use weather data from a number of satellite and land-based services.
Metcheck started life in 1998. They have a small group of meteorologists at seven locations around the world. Most of their data is done using computer models from around 1GB of data collected each day.
Weather Forecasting Evaluation
This test is a fairly simplistic one. From the 7th – 10th March we collected a reading from each of the seven services websites. We were going to use the actual weather first thing in the morning, lunchtime and just before sunset on Sunday 10th to assess the accuracy of the forecasts. We were only going to look at two aspects of weather that is important to photography – the amount of cloud cover and the amount of rain.
In evaluating how good the forecasts were if the service stated that it would rain at a particular point in time and it did then it would get 1 point. Similarly, if they predicted dry and it was dry then they also got a point. Any other result and they scored zero points. We took a similar approach for cloud cover with options of either clear, part cloudy, mainly cloudy and overcast. Again, if the right result was predicted the service scored a point, if it was something different then it scored zero points.
We can then look at how each service performed by comparing the points scored at 3 days, 2 days, 1 day and on the day of the comparison. We’ll also look at how they performed overall. We’ll rank each day performance by adding the scores taken over the three time periods for each day – this would mean a service could achieve a maximum score of 6 for each day (3 correct rain forecast and three correct cloud forecasts). The overall performance would be judged out of 24 (i.e all four days performances added together).
Three-day-out performance
This, first day of evaluation, should give a good view as to what the weather will be like in 3 days’ time. It’s quite interesting to see that two of the weather services, Yr and Metcheck had pretty good accuracy with 5 points out of 6. However, Weather Underground only scored 1 point out of the six readings which were for the eventual dry end to the day.
| Rain Accuracy | Cloud Accuracy | Total | Rank | |
| Darksky.Net | 2 | 2 | 4 | 3 |
| BBC Weather | 1 | 2 | 3 | 5 |
| Met Office | 2 | 2 | 4 | 3 |
| Weather Underground | 1 | 0 | 1 | 7 |
| Accu weather | 1 | 2 | 3 | 5 |
| Yr.no | 3 | 2 | 5 | 1 |
| Metcheck | 2 | 3 | 5 | 1 |
Two-day-out performance
As we move closer to the evaluation day it would be hoped that the accuracy of prediction would improve. It certainly did for Weather Underground who now joint topped the charts. Metcheck also retained its good performance also with five out of six points. They were also joined by AccuWeather. With the exception of BBC Weather, all the services scored over 50% two days out. This was the most accurate day for predictions in advance of the workshop date.
| Rain Accuracy | Cloud Accuracy | Total | Rank | |
| Darksky.Net | 2 | 2 | 4 | 4 |
| BBC Weather | 1 | 2 | 3 | 7 |
| Met Office | 2 | 2 | 4 | 4 |
| Weather Underground | 3 | 2 | 5 | 1 |
| Accu weather | 3 | 2 | 5 | 1 |
| Yr.no | 2 | 2 | 4 | 4 |
| Metcheck | 2 | 3 | 5 | 1 |
Prior day performance
Rather disappointingly, the prediction accuracy of all the services got worse the day before the workshop. Two of the services that had underperformed, Darksky and Met Office became the most accurate services with 5 out of 6 points.
| Rain Accuracy | Cloud Accuracy | Total | Rank | |
| Darksky.Net | 3 | 2 | 5 | 1 |
| BBC Weather | 1 | 2 | 3 | 5 |
| Met Office | 3 | 2 | 5 | 1 |
| Weather Underground | 1 | 2 | 3 | 5 |
| Accu weather | 2 | 2 | 4 | 3 |
| Yr.no | 1 | 2 | 3 | 5 |
| Metcheck | 2 | 2 | 4 | 3 |
On day performance
Not surprisingly, as we got to the morning of the forecast, all the services had a good performance with the exception of Yr. Metcheck and Weather Underground both predicted perfect results scoring 6 out 6 – a result that I would have hoped to have seen across the board. Based on this very small sample it would seem that even on the morning of a shoot, some services can’t give a good indication of the weather that is expected!
| Rain Accuracy | Cloud Accuracy | Total | Rank | |
| Darksky.Net | 3 | 1 | 4 | 5 |
| BBC Weather | 2 | 3 | 5 | 3 |
| Met Office | 2 | 3 | 5 | 3 |
| Weather Underground | 3 | 3 | 6 | 1 |
| Accu weather | 1 | 3 | 4 | 5 |
| Yr.no | 1 | 2 | 3 | 7 |
| Metcheck | 3 | 3 | 6 | 1 |
Overall Performance
By adding together the scores for all four individual readings we can establish the best prediction service. Again, remember the health warning, this is only one set of data from one location. To get a more definitive result it would need to be repeated several more times at different locations.
The best service in this trial was Metcheck with 20 points out of 24. It did make the top position on three of the four days and seems to be a worthy winner. Two points behind in second place were the Met Office. The final step of the podium was taken by Darksky with 17 points.
The full results are listed below;
| Rain Accuracy | Cloud Accuracy | Total | Rank | |
| Darksky.Net | 10 | 7 | 17 | 3 |
| BBC Weather | 5 | 9 | 14 | 7 |
| Met Office | 9 | 9 | 18 | 2 |
| Weather Underground | 8 | 7 | 15 | 5 |
| Accu weather | 7 | 9 | 16 | 4 |
| Yr.no | 7 | 8 | 15 | 5 |
| Metcheck | 9 | 11 | 20 | 1 |
Whilst it was disappointing to postpone the workshop, the actual weather on the day vindicated this decision. At varying times of the day, there was dull and overcast clouds, heavy rain, and very occasional glimpses of sun. Sometimes, these conditions can get some great photos, but only if you are prepared to stay in wet weather until the rain and clouds clear for a few minutes. This wouldn’t have been fun for the attendees and they all agreed that it was the right call to postpone.
If you have enjoyed this blog then you could sign up to our monthly newsletter where you’ll be able to catch up on past blogs, receive interesting photography news and have special members-only content.