Skip to main content

Building an accessible agricultural data community with the National Agricultural Producers Data Cooperative

By Raleigh Butler

Romaine lettuce crop grown on a city farm in Moscow. Photo by Petr Magera.
Photo by Petr Magera/Unsplash

Entities around the world gather data focused on various aspects of agriculture. Unfortunately, this information is not always accessible or easily available for those who need it. The National Agricultural Producers Data Cooperative (NAPDC) project recognizes that agriculture is a keystone of society and a critical piece of national solutions to climate-related challenges. The NAPDC, with support from the United States Department of Agriculture (USDA), aims to enable agricultural producers to benefit from the massive amounts of data generated by members of their community. As the NAPDC site states, the goal of the project is to create a “blueprint” for a national data framework where agricultural entities “can store and share data . . . to maximize their production and profitability.”

With enough available data and methods to extract relevant information, national agricultural systems can become more efficient and profitable. The framework being developed by the NAPDC will include data from many types of agricultural contexts and agricultural institutions, first and foremost the producers that drive agricultural productivity. Making the system diverse yet robust while safeguarding farmer privacy will result in a more reliable set of data for the entire agricultural community.

The NAPDC project emphasizes providing resources to community partners through webinars and seed grants in order to “identify needs and opportunities as well as challenges in physical infrastructure, education and human resources, and critical use cases” critical to the success of a future data framework. The project recognizes that a secure framework is necessary to protect privacy and governance information; these aspects will be carefully considered. The project also recognizes the importance of land-grant institutions and agricultural extension in the successful deployment of any framework.

The NAPDC project has a seed grant program to support development of community activities, with a deadline of June 1, 2022. It will be granting 4–6 awards; complete guidelines are listed on the site here. The grants will not be limited to principal investigators at universities; rather, any institution eligible for USDA funding may apply. As stated on the website, “individuals willing and qualified to lead representation for a national or regional agroecosystem are encouraged to apply.”

“The work of the NAPDC aligns well with the Digital Agriculture community of the Midwest Big Data Innovation Hub,” said MBDH Executive Director John MacMullen. “We anticipate integrating findings from our Community Data Needs Assessment (Community DNA) activities, which are helping to understand the data needs of stakeholders across the food supply chain, with the work of the NAPDC. We also look forward to partnering with the NAPDC team on our agricultural data work with the IEEE Standards Association and other partners.”

Jennifer Clarke, lead PI of the NAPDC project and faculty at the University of Nebraska–Lincoln, hopes the project serves as an initial step towards a national framework. “This project represents the willingness of the USDA to listen to agricultural producers and support the data needs of producer communities,” said Dr. Clarke. “This project provides producers and stakeholders with a vehicle for communicating their challenges related to data, and provides educators and researchers with a vehicle for proposing solutions to these challenges.”

The NAPDC will host an All-Hands Meeting in the spring of 2023 at the University of Nebraska–Lincoln that will highlight the work of the NAPDC and discussions of specific areas for future USDA investment. Interested members of the community can sign up for the project listserv through the project website (https://www.agdatacoop.org/) to receive updates about this meeting as well as project information.

Get involved

Do you have an agricultural data success story or case study to share from your organization? Contact the Midwest Big Data Innovation Hub if you’re aware of other people or projects we should profile here. The MBDH has a variety of ways to get involved with our community and activities.

The Midwest Big Data Innovation Hub is an NSF-funded partnership of the University of Illinois at Urbana-Champaign, Iowa State University, Indiana University, the University of Michigan, the University of Minnesota, and the University of North Dakota, and is focused on developing collaborations in the 12-state Midwest region. Learn more about the national NSF Big Data Hubs community.

New MBDH Community Development and Engagement partners

By Qining Wang

The Midwest Big Data Innovation Hub (MBDH) recently partnered with multiple institutions in the region for new data science activities under its Community Development and Engagement Program. This program incubates new projects and provides support to help them grow.

In the last proposal cycle, the MBDH Seed Fund Steering Committee selected three projects to support, led by the Tribal Nations Research Group (TNRG), St. Catherine University, and Trinity Christian College.

TNRG Digital Agriculture Meeting

The TNRG, together with the University of North Dakota and Grand Farm/Emerging Prairie, will host a one-day workshop in 2022, at the Microsoft Business Center in Fargo, North Dakota. This workshop will connect tribal colleges and universities working with their local tribal governments to extend digital agriculture and educational opportunities to Native farmers.

Approximately 30% of the nation’s Native population and 20 of the 37 of the nation’s tribal colleges and universities are located in the MBDH service area. Because of this, the MBDH is well-positioned to engage tribal stakeholders on issues related to Data Science Education and Workforce Development. This is especially true in the context of Digital Agriculture, where many of these institutions are working with their local tribal governments to extend agricultural programs and educational opportunities to Native farmers.

Tribal communities have not had the dedicated capital for building a resilient and sustainable infrastructure for harnessing food on their lands for a long time. The lack of such infrastructure creates food insecurity that can be detrimental to Indigenous peoples. In addition, due to climate change, it is crucial to build sustainable farming practices that can provide sufficient food and preserve the ecosystem everywhere in the long run.

One way to realize optimal farming practices is to incorporate digital agriculture, which integrates digital technologies into crops and livestock management. Technologies such as machine learning and big data analysis tools can improve agricultural production while minimizing the harm to the ecosystem. For instance, by correlating multiple parameters related to crop growth using machine learning, farmers can better predict crop yield based on other parameters such as nutrients in the soil, weather, and fertilization. Those technologies can therefore make information on ecosystems, crops, and animals more findable and interpretable to farmers.

However, implementing digital agriculture on tribal lands involves extra layers of nuance. Data scientists and agricultural experts must conduct digital agriculture research in tribal regions under proper data sovereignty standards, such as the CARE Principles for Indigenous Data Governance. Indigenous peoples are entitled to know what data is collected and how data scientists use and analyze their data. The data should enable Indigenous peoples to derive benefit from any fruits of the research involving tribal communities.

This workshop will serve to increase the accessibility of digital agriculture in Native communities, emphasizing respecting the culture, traditions, and sovereignty of the Native people. In addition, this workshop will enlist more tribal stakeholders nationwide for broader engagement in digital agriculture, potentially developing a Data Science Workforce Development and Education proposal for Native communities. Anita Frederick, the President of TNRG, will lead this workshop and present the importance of Data Management and Data Sovereignty.

“Outreach to Indian tribes is often difficult for non-tribal entities and individuals,” Frederick said. “As a direct result, tribal populations are often left out of initiatives that could help to address some of the economic, health, and other societal conditions that tribes face. Clearly, American Indian citizens must have access to the opportunities envisioned in the Big Data Revolution. The proposed project is a first step in helping to close the growing Big Data gap that is emerging between Indian country and the rest of the nation.”

St. Catherine Data Science Boot Camp

MBDH will also support a data science program “created by women for women” at St. Catherine University (aka St. Kate’s), one of the USA’s largest private women’s universities, located in St. Paul, Minnesota. This program aims to cultivate a new generation of women and historically underrepresented data scientists. In addition to teaching data science and data analytic principles, this program will also raise students’ awareness of using data science in ethically, socially, and environmentally just ways.

Introduced in the fall semester of 2018, the data science program at St. Kate’s reaches both current and prospective students of the University. Monica Brown, the Mary T. Hill Director of Data Science at St. Kate’s, will lead the program’s two initiatives in 2021-2022. Working alongside her colleagues at St. Kate’s for over 13 years, Brown aspires to make data science and data analytics principles accessible to every student in the St. Kate’s community.

Brown will launch a one-week Data Science Boot Camp in the summer of 2022. This boot camp will provide hands-on coding experience to middle- and high-school students, particularly those historically excluded from data science. In addition, Brown will invite data science professionals to speak about future career opportunities. Overall, this program aims to enable younger students to envision themselves as future data scientists and to elicit their passion for coding and data science. The lessons learned organizing this event will be shared with others who wish to do so with their own student populations.

“St. Kate’s is grateful for the partnership with MBDH towards the support of a boot camp,” said Brown. “We very much look forward to bringing younger students onto our campus to encourage and empower them through data science activities.”

Trinity Data Science for Social Good Workshop

The third project to be incubated under the MBDH’s Community Development and Engagement program will be an annual workshop and conference on Teaching with Data for Social Good (DSG) in summer 2022. DSG addresses the importance of teaching data science for positive social impact, and this conference serves as an opportunity that encourages teaching faculty to include DSG in their curricula proactively.

Trinity Christian College, a faith-based institution located on the outskirts of Chicago, will host this meeting. The workshop chair will be Dr. Karl Schmitt, an assistant professor in the Data Analytics department at Trinity and the coordinator of the Data Analytics program.

The meeting format resembles that of regional professional society meetings, consisting of a workshop, keynotes, and contributed talks. To provide more practical assistance to teaching faculty incorporating DSG, faculty will directly generate teaching materials that include DSG in the primary workshop sessions. Additionally, faculty will also have a chance to practice teaching DSG by actively advising student teams participating in a colocated datathon. In this student competition, student teams will use data science to solve practical problems.

“An important component of increasing persistence and success for our current generation of students is connecting their coursework to meaningful change or outcomes,” Schmitt said. “Through the Workshop on Data for Good in Education, the MBDH will be supporting faculty in developing their teaching to better incorporate the Data for Social Good (DSG) movement. This provides a natural connection to relevance with grass-roots level improvements in our society while promoting the broad applicability of data science.”

Beyond these outcomes, Schmitt said, “the workshop will be a professional development opportunity for all instructors seeking to more deeply engage their students through meaningful social good projects within a classroom setting. It will inspire, educate, and most importantly, allow faculty the chance to share, and prepare, materials for use within their own teaching context.”

Get involved

Learn more about other Community Development and Engagement partnerships, and contact the MBDH if you have an idea for a project to help build the data science community in the Midwest.

Contact the Midwest Big Data Innovation Hub if you’re aware of other people or projects we should profile here, or to participate in any of our community-led Priority Areas. The MBDH has a variety of ways to get involved with our community and activities.

The Midwest Big Data Innovation Hub is an NSF-funded partnership of the University of Illinois at Urbana-Champaign, Iowa State University, Indiana University, the University of Michigan, the University of Minnesota, and the University of North Dakota, and is focused on developing collaborations in the 12-state Midwest region. Learn more about the national NSF Big Data Hubs community.

Agroterrorism: Cybersecurity Incidents Affect Agriculture and Water

By Raleigh Butler

You may not think that agriculture and cybersecurity, both themes of the Midwest Big Data Innovation Hub, are linked, but recent events demonstrate there are connections between the two that pose risks to our food security.

The “food and agriculture” industry is publicly defined as a critical infrastructure sector by the U.S. Department of Homeland Security. The Cybersecurity & Infrastructure Security Agency (CISA) states that food and agriculture is one of sixteen essential critical infrastructure sectors that provide “the essential services that underpin American society and serve as the backbone of our nation’s economy, security, and health. We know it as the power we use in our homes, the water we drink, the transportation that moves us, and the communication systems we rely on to stay in touch with friends and family.” Those statements highlight the urgency of building robust cyberinfrastructure to prevent massive disruptions to crucial public services.

A recent cyberattack targeting an Iowa-based agriculture company called New Cooperative illustrates the severity and consequences of those incidences. The group claiming responsibility—BlackMatter—deals in blackmail, Reuters reports. The hackers from BlackMatter locked New Cooperative’s access to data that support the food supply chains and detail the feeding schedule of the livestock. In order to get access to the decryption key for its data and reinstate their farming activities, New Cooperative was ordered to pay $5.9 million.

As Bobby J. Martens, an associate professor of Economics at Iowa State University was quoted as saying, “This event wasn’t long enough to cause a change in the commodity price, but certainly it will have ramifications in terms of the food supply system. If they do it to this company, they could do it to one of the majors. They can block the food chain. They attacked in the heartland of all agriculture. It’s a new form of terrorism.”

Regardless of the source, and whether it is purposeful or accidental, a failure in any other critical sector could be life threatening for US citizens. For example, Water and Wastewater Systems is a related sector on CISA’s list, and in fact, water system attacks did occur early in 2020, the most prominent being the Oldsmar, Florida attack of February 16. While the breach nearly allowed a mass poisoning to occur, the mayor viewed the event as a “success.” According to ProPublica, cybersecurity experts view the breach not as a success, but instead as a “frightening near-miss.” Retired Admiral Mark Montgomery, a panelist on the MBDH Water Data Forum webinar on water and cybersecurity in May 2021, was quoted as saying, “Frankly, they got very lucky. They averted a disaster through a lot of good fortune.”

Nontechnical companies are extremely vulnerable to cyberattacks. According to the 2020 state of ransomware report, manufacturing, government, services, and healthcare are among the top sectors prone to cyberattacks. This link leads to this report from a company called BlackFog, a leading company in ransomware protection.

Moving forward, it is possible for businesses and governmental sectors to make cybersecurity an integral part of their practices. Even seemingly trivial data maintenance, such as regularly backing up data in multiple storage devices and encrypting data during transfer, can improve data security in the long run. The key is to operate under the mindset of protecting data and to be more intentional about data protection at any point. The U.S. National Institute of Standards and Technology (NIST) and CISA developed the NIST Cybersecurity Framework, a comprehensive approach to security for critical infrastructure, and there are subsets of that work to support small businesses and other organizations with cybersecurity risks that may not have extensive resources.

On the management level, designated information security officers can build more secure databases and data management systems. The information security officers can also perform routine testing for weaknesses in the existing systems. They could also work with the risk managers to develop preventative measures in case of cyberattacks. Other preventive measures include purchasing cyber insurance.

An additional benefit of developing systems for monitoring and collecting data is the ability to assess the impact of other external events. We previously published a story on how researchers were assessing the spread of COVID-19 by examining the relative levels of the virus in wastewater systems. Since many infrastructure systems, such as agriculture, water, and food, are an interconnected web of dependencies, threats to one can have cascading impacts across others. For academic organizations that manage research data repositories, the MBDH and its partners developed a guidance document on data security for open science, through our Trustworthy Data Working Group.

Get involved

Do you have a cybersecurity success story or case study to share from your organization? Contact the Midwest Big Data Innovation Hub if you’re aware of other people or projects we should profile here, or to participate in any of our community-led Priority Areas. The MBDH has a variety of ways to get involved with our community and activities.

The Midwest Big Data Innovation Hub is an NSF-funded partnership of the University of Illinois at Urbana-Champaign, Iowa State University, Indiana University, the University of Michigan, the University of Minnesota, and the University of North Dakota, and is focused on developing collaborations in the 12-state Midwest region. Learn more about the national NSF Big Data Hubs community.

Climate Change Affecting Crops in Iowa

By Raleigh Butler

In 2010, the University of Minnesota received a grant from the National Science Foundation to study climate change using data-driven methods. The project included Midwest Big Data Innovation Hub co-PI Shashi Shekhar and a team of researchers from across the country. The research led, in part, to explorations of connections between food, energy, water, and climate change.

Because greenhouse gases contribute heavily to climate change, activities that contribute to their release are becoming more divisive with time. There’s no doubt that the food we eat is becoming an increasingly political statement. According to the 2019 Environmental Protection Agency report, agriculture was responsible for 10% of all greenhouse gas emissions, amounting to 650 million metric tons of CO2. A quarter of those emissions (about 2.5% of all greenhouse gas emissions) come from livestock before they are butchered.

The Coupled Model Intercomparison Project 3, a project of the World Climate Research Programme, predicts when the global average temperature will increase by 2°C. The approximately 0.75°C increase in temperature since 1950 has caused a huge increase in natural disasters. This can be seen by an increase in hurricanes, such as Katrina, and the melting of the polar ice caps, among other issues.

According to the graphic from their report, the global average temperature has already increased by about 1°C (1.8°F) relative to preindustrial levels, and will continue rising to as much as 7°C in some regions by the end of the century.

Climate change is the culprit behind many natural disasters, as more than 170 scientific reports covering 190 extreme-weather events found that around two-thirds of extreme-weather events likely originated from, or were exacerbated by, anthropogenic hazards.

How does this apply to the Midwest? Let’s look at Iowa, where over 90% of its land is used for agriculture. In recent years, extreme-weather events have wreaked havoc on crops.

2020: Inland Hurricanes

Farmers and agricultural specialists were worried in August 2020 when portions of Iowa experienced derechos. Pronounced deRAYchos, these are widespread, long-lived thunderstorms mixed with 100–130 mph winds. According to the National Weather Service, a derecho like this was “a roughly once-in-a-decade occurrence” in the Midwest.

These immensely strong storms destroyed crops and decreased crop output for the season in Iowa. According to the power-outage map published by the University of Wisconsin–Madison’s Cooperative Institute for Meteorological Satellite Studies (CIMSS) below, A quarter of the counties in Iowa caught the worst of the storm. All the affected counties were in the central-east portion of the state.

2021: Drought

There were high hopes that 2021 would bring a better crop return. However, when agricultural scouts crossed Iowa in mid August, they found that the state was suffering from extreme drought.

Although derechos and rain-damaged fields were no longer the center of concerns, 2021 has brought high levels of drought. According to the U.S. Drought Monitor, on August 17, 2021, 79% of Iowa was impacted by some degree of drought.

National Drought Mitigation Center map of the 2021 drought in Iowa
The U.S. Drought Monitor is jointly produced by the National Drought Mitigation Center (NDMC) at the University of Nebraska–Lincoln, the United States Department of Agriculture, and the National Oceanic and Atmospheric Administration. Map courtesy of NDMC.

The areas were being scouted out ahead of time for the upcoming 2021 Pro Farmer Crop Tour. Scouts on crop tours have the job of evaluating likely crop production in each region. For more information on crop tours, visit this link.

2021: Storms

Drought became an afterthought just days later. On August 24, 2021, the Midwest experienced multiple storms. Although the severity of the storms did not come close to derechos, they still left behind large paths of downed corn and soybeans. On August 28, 2021, South Dakota and southwest Minnesota even experienced baseball-sized hail.

According to Iowa State University (ISU) Extension Field Agronomist Terry Basol, “The storms hit northeast Iowa farms pretty good, honestly.” Basol said, “It’s amazing the scope of the crop damage,” he continued, concerned about the pace of harvest and crop quality.

Unfortunately, the rain has come too late for many crops, and on top of that, some areas are even flooding. One person, Iowa State University Extension crop specialist Angie Rieck-Hinz, said, “The crop is highly variable. Crop conditions are literally all over the place.”

What’s to Come?

Amidst all these natural disasters and climate change, what can be expected for the future of agriculture in Iowa? In April 2021, the Environmental Defense Fund commissioned KCoe Isom, an agricultural consultancy, to model the potential climate change impacts on Iowa corn, soy, and silage production over the next two decades. According to that site, “Iowa farmers could see statewide gross farm revenues reduced by as much as $4.9 billion per decade. Because with climate change agricultural prices are likely to rise, relative to without climate change, the impact to gross farm revenues from yield impacts will be offset to some degree by higher prices.”

Unfortunately, the increase in climate change (and resulting natural disasters) is likely to continue reducing levels of crop production. This will result in an increase in food prices where those crops are sold, affecting consumers across the country.

The roles for data science and related research around climate and agriculture are growing: in September 2021, the National Science Foundation funded a new multidisciplinary institute led by the University of Illinois, called I-GUIDE, which is focused on better understanding the risks associated with climate change.

Get Involved

Contact the Midwest Big Data Innovation Hub if you’re aware of other people or projects we should profile here, or to participate in any of our community-led Priority Areas. The MBDH has a variety of ways to get involved with our community and activities.

The Midwest Big Data Innovation Hub is an NSF-funded partnership of the University of Illinois at Urbana-Champaign, Iowa State University, Indiana University, the University of Michigan, the University of Minnesota, and the University of North Dakota, and is focused on developing collaborations in the 12-state Midwest region. Learn more about the national NSF Big Data Hubs community.

Machine Learning: Farm-to-Table Workshop

by Keith Hollenkamp –

In April, the MBDH teamed up with the International Food Security at Illinois (IFSI) to host the Machine Learning: Farm-to-Table Workshop. The workshop brought together domain scientists to stimulate new data-driven R+D activity at the intersections of the Agriculture, Bioinformatics, Food-Energy-Water, and Food Security communities.

Read More

The Machine Learning: Farm-To-Table Workshop

The Midwest Big Data Hub (MBDH) is partnering with the International Food Security at Illinois (IFSI) group at UIUC to bring together domain scientists from the Agriculture, Bioinformatics, Food-Energy-Water, and Food Security communities, along with computational experts. The objective of this workshop is to stimulate new data-driven R+D activity at the intersections of these communities. The meeting will be structured to enable new cross-community interactions and initiate grant proposals or publications.

Read More

Food and Data Workshop: Interoperability through the Food Pipeline

September 12-13, 2016
University of Illinois at Urbana-Champaign

The increasing ability to capture data at the level of individual agricultural fields, individual culinary recipes, and individual food waste digesters is allowing analytics-based optimization within the distinct industries responsible for producing, transporting, trading, storing, processing, packaging, wholesaling, retailing, consuming, and disposing of food. Yet addressing the pressing national/global challenges in food security due to climate change, as well as public health challenges such as obesity and malnutrition, requires optimization across the food pipeline. The Food and Data Workshop: Interoperability through the Food Pipeline, September 12-13 in the CSL Auditorium (B02), is concerned with understanding the relationship between data and food writ large, with a particular focus on questions of interoperable data ontologies, privacy, and analytic insights.

For more information and to register go to https://publish.illinois.edu/food-and-data-workshop/.

Digital Agriculture Spoke All-Hands Meeting – May 16-17, 2016

Videos of the presentations are now available!

The 2016 Digital Agriculture Spoke All-Hands Meeting to be held on May 16-17 at the Scheman Building, Iowa State Center, Ames, Iowa. The Digital Agriculture Spoke of the Midwest Big Data Hub is devoted to building partnerships and resources that will address emerging Big Data issues in the agricultural ecosystem.

Stakeholders from academia, industry, government, and other organizations will engage in interactive discussions about the partnerships and resources that will be needed to address the challenges in collecting, managing, serving, mining, and analyzing rapidly growing and increasingly complex data and information collections to create actionable knowledge and guide decision-making in agriculture.

Events will include presentations by Midwest Big Data Hub and national leadership; industry panel presentations and Q&A; participant lightning talks; and breakout sessions to discuss existing projects and to develop ideas and partnerships for new projects; and a poster session and reception.

Early career researchers, post-docs, graduate students, and undergraduate students are encouraged to attend. There is no registration fee for this meeting.