I think structured web data is a very promising data source for future academic research. It has a very broad coverage of companies but also in terms of frequently updated web information.
Prof. Dr. Michael D. König, associate professor at the Vrije Universiteit Amsterdam (VU Amsterdam) and affiliated with the ETH Zürich Switzerland, did research in collaboration with Dataprovider.com in which he used structured data from company websites to analyze the effects of the pandemic on businesses across time and different regions.
"My research looks at how companies are affected by different types of shocks. In the past, we have mostly studied how companies are affected by technological change. More recently, we have looked into how companies are affected by the COVID-19 pandemic and we are looking into new ways how to measure this effectiveness by firms. One very promising way seems to be the analysis of company websites. In this respect, we were very happy to collaborate with Dataprovider.com to get information about company websites and how they write about the recent COVID-19 pandemic. So my research tries to use company websites as a new data source to measure how companies are affected by the COVID-19 pandemic and the different policy measures that have been implemented, for example, across different regions, sectors and also across time. For the purpose, we try to look at websites of companies and try to understand how companies write about the pandemic and whether we can get some information out of this.
And we have done some preliminary analysis using the Dataprovider.com search interface which made it very convenient for us using a keyword-based approach. And we could, for example, investigate and understand how firms in Switzerland were affected during different times of the pandemic. In Switzerland in 2020 and 2021, there were two waves and we could see that in some sectors, firms were exposed more broadly to production shocks during the first wave of the pandemic while this was less of a problem in the second wave of the pandemic.
So about a year ago, we started the collaboration with Dataprovider.com using the website content to understand how or which companies mentioned the keywords corona or one of its synonyms. And then we tried to see which sectors and which countries across the world were mentioning corona the most. And we provided simple, descriptive statistics and rankings over sectors and countries and how they changed over the first month in the year 2020.
Based on that, we were then interested to see in which context companies mentioned the corona pandemic on their websites. So we developed a set of keywords that allowed us to identify whether companies mentioned corona in the context of a production problem, a supply problem or a demand problem. And we were then able to better understand the specific problems companies were facing: which sectors are most affected by that and how that has changed across time and also across regions?
We did that for a specific country, Switzerland, where we also had alternative data sources to kind of compare the outcomes from this web based approach. And we did find that the website information could be used to identify production problems on company websites. So my future research in collaboration with Dataprovider.com will try to look into how the effectiveness of companies measured through mentioning the COVID-19 pandemic on their websites differs across countries.
We started with an analysis of a specific country, Switzerland. In the next step, we would like to expand this analysis to other countries because this would allow us to evaluate the policies that have been implemented by different countries and how the pandemic has spread across different countries over time.
Another direction that we would like to explore is to see the frequency with which companies update their websites. This can be measured by an index that Dataprovider.com has developed, the Heartbeat indicator. So is it the case that the COVID-19 pandemic triggered certain firms to update and expand their websites? Or is it the case that companies that already have highly or frequently updated website were less affected by the pandemic and were, in this sense, more resilient to this shock?
I believe that structured web data can be a very important data source for academic research because it has a very high coverage. When we look at company websites and almost every company has a company website nowadays, this would allow us to get an almost complete picture of the entire economy of a country or even worldwide.
I believe that structured web data can be a very important data source for academic research because it has a very high coverage. When we look at company websites and almost every company has a company website nowadays, this would allow us to get an almost complete picture of the entire economy of a country or even worldwide. And it also allows us to have highly, frequently updated information about how companies are doing if they use their website as a communication tool to the outside world. So we have a very broad coverage that you cannot find in other data sources when it comes to including our companies and always in terms of highly, frequently updated information that we can find in this way."