Data Science In Action: Harvesting the Benefits of Data Science in Philippine Agriculture

According to a famous saying, “you reap what you sow.” This very old proverb is true for almost any activity in life. Whether you talk about school or career, you generally get good results if you work hard.

Ironically though, the saying does not apply to many Filipino farmers.

According to Rappler, many farmers in the Philippines can barely survive because of the many problems they face. COVID-19, typhoons and the massive importation of rice gives farmers a hard time in either maximizing output or finding profitable markets for their produce.

While we are not able to solve these issues directly, data science applications that are being deployed at large tech companies can also empower local solutions built for farmers. In this article, we will highlight the work of Eskwelabs Data Science Fellows with agritech startup Mayani. Their capstone project provides an example of how data can be used for good. 

Co-Creating a Recommendation Engine

Our Data Science Fellowship is a 15-week, project-based upskilling program for aspiring data professionals. At the end of the 15 weeks, each cohort or batch is expected to design and implement a capstone project before they can graduate.

‍A group of Eskwelabs Data Science Fellows completed a data science project to empower Mayani, an agritech startup. Eskwelabs partnered with Mayani to not only to provide our learners industry-relevant experience, but also to enable us to pursue our vision of “data for good.” After all, Mayani exists to ensure small farm-holders earn reasonable income through an online market platform. Therefore, our tie-up with this startup is beneficial to the agricultural sector, which is the backbone of our country’s economy. 

Mayani is better able to pursue its advocacy for farmers thanks to the data science skills of Data Science Fellows Charity Benignos, Christopher Louie Gemida, Renzo Luis Rodelas, Andrew Justin Oconer, and Matthew Antoine Tomas. They worked together and leveraged their budding data skills to create a recommendation engine that helps Mayani match the right food products for the right customer segments.

‍

The Challenge

Have you ever wondered why many of your video streaming services and online shopping apps seem to “know” what shows or products you like, even before you buy them? Though this capability seems to border on the supernatural, it uses a software algorithm called a recommender engine.

Mayani, being an online seller of agricultural products, is also seeking to leverage a recommender engine.  This is where Eskwelabs Data Science Fellows come in.

The Solution

Through a data-driven analysis of Mayani’s current recommender app and the startup’s sales and customer information, our Data Science Fellows were able to improve the way the agritech startup recommends the right farm products for the right customer.‍

These were the steps they followed:

1. Business Understanding - Our data science learners spoke with the key leaders of Mayani to determine the business needs. This is also the stage where the Fellows tried to understand the basics of how Mayani’s current recommender app works.

2. Data Preparation -  Mayani’s volumes of customer data and sales figures were cleaned (processed so computers can understand the data) and subjected to Exploratory Data Analysis (EDA). Customer data includes order dates and money spent on certain farm goods. On the other hand, the startup’s sales figures were broken down into monthly and weekly periods, with monthly sales statistics covering the period from May 2019 to July 2021. The cleaned sales data were then used in the modeling phase.
3. ‍Apriori Algorithms - After K-Means clustering, our Data Science Fellows designed two types of algorithms which will power their proposed recommender engine. The first algorithm, which is called the “General Apriori Algorithm,” is executed by the engine whenever a new customer or a buyer who has no account with Mayani tries to buy a farm product. The other algorithm, called the “Modified Apriori Algorithm,” is applied to customers who fall in one of the three segments from the KMeans clustering.
   ‍To illustrate, if a new customer buys some products from Mayani, the proposed recommender app will execute the General Apriori Algorithm to recommend agricultural products to that consumer. If a client has been buying frequently from Mayani for a certain amount of time (the thresholds or parameters of frequency and transaction history are set by our learners), the proposed recommender will show farm goods in accordance with the instructions of the Modified Apriori Algorithm.
4. ‍The Impact  - Based on the team’s evaluation of the two apriori algorithms, they were able to design a new recommender engine for Mayani and came up with the following recommendations for the agritech startup:
   The team’s Market Basket Analysis can be used by Mayani to suggest product bundles based on what customers frequently buy.
   The Fellows’ RFM customer clustering process can be used by the startup to design personalized marketing campaigns and promotions.

Tools of the Trade

Our data science learners used the following software to build their improved recommender for Mayani: 

• Jupyter Notebooks - An app that can run Python and other select programs through the execution of files called “notebooks.” This app can be accessed through a web browser or on an offline computer.

• Pandas - An open source program which was used to create models for the Exploratory Data Analysis and data cleaning in the Data Preparation stage. The software was run on Jupyter.

• Python 3 - A recent iteration of a high-level programming language that was utilized for the initial survey of data. Jupyter was also used to run this version of Python.

• SciKit Learn - Python-based machine learning software that was used for customer segmentation through KMeans clustering. SciKit Learn was run on Jupyter during the data preparation and modeling stages.

• UbiOps - A web-based app that was used to run the models that were built using Pandas, Python 3 and SciKit Learn.

• ReactJS - A Javascript library used by our data science learners to code their recommender app.

• Vercel - The web-based software which deployed the ReactJS-based recommender app.  

From the way our data science learners helped improve Mayani’s business, it is clear that data science has the power to reach out even to seemingly unrelated sectors like agriculture. However, some of you may ask, what prospects do data science graduates have in the agricultural and agribusiness sectors? Do the prospects for this kind of data-driven work look good?

‍

What’s Next?

Data scientists can expect a bright future, even those who aspire to empower agribusinesses like Mayani. According to a 2020 article in the Philippine Daily Inquirer, the Philippine government aims to support the Department of Agriculture’s eKADIWA online marketing platform. This e-commerce project is the government equivalent of the Mayani business model. 

It is clear, therefore, that data scientists who help facilitate online agricultural transactions have a place both in the public and private sectors.

Data science combines fields like machine learning and statistics to derive insights or create software from different types of data. The prototype built by our learners, which extensively used machine learning to process Mayani’s large amount of information, proves that data science can indirectly help rehabilitate the Philippine agricultural sector by empowering agritech startups and other similar organizations. 

A recommender engine, like the one designed for Mayani, is an example of the power of data science in enabling computers to “know” what you or other customers want to buy in the future. The ability to empower machines to “learn” what buyers want is one of the applications of data science.

‍

Empower Yourself or Your Business Through Data Science

We live in a world that swims in data. Whether you’re working for a big corporation or a small grocery store, information like sales figures, income, etc. can be found there. In the 21st century, the ability to identify, collect, and meaningfully use data is one of the most valuable skills for success.

Whether you want to use machine learning to help agritech startups, or pursue your passion, data science is among the best career paths you could consider. After all, since many endeavors these days have some form of data in them, data science skills may prove useful as you pursue your dream.

As the economy and jobs become more digital, the use of data to make decisions is on the rise. And anyone equipped with the right data science skills will be able to make better decisions as well as be resilient to automation. 

Eskwelabs supports those who want to pursue whatever data-driven dream they have for themselves or their organization. Our Data Science Fellowship is a part-time data upskilling program which features engaging, industry-relevant, and project-based data education. Learners in our Fellowship create data science projects with the help of an industry mentor. Our mentors complement our instructors in ensuring that no learner gets left behind as they acquire data skills in an interesting and supportive environment. 

To top it all off, we are taking our data science education to the next level by introducing paid apprenticeships. Our Data Science Fellowship Industry Apprenticeship gives our learners the opportunity to extend the projects they work on in the bootcamp while getting paid to do them. We are partnering with Accenture and the startups of the Asian Institute of Management - Dado Banatao Incubator (AIM-DBI) to offer this to our next cohort of Data Science Fellowship learners. Sign-up to learn more in the Recommended Reading section below. 

If you are a company looking up to bring data talent to your team or build up the skills of your existing team, you can reach out to us at [email protected]

The Eskwelabs Data Science Fellows who designed Mayani’s new recommender engine have proven a lot of things. Their work shows not only their data science capabilities, but also the power of learners, startups and Eskwelabs to work together to pursue data for good.

RECOMMENDED READING

Interested in our Industry Apprenticeship? You may click any of the links below to learn more, according to your program of choice.

• Data Science Fellowship
• Data Analytics Bootcamp

You too can co-create your own cutting-edge recommender engine. Click here to sign up for our Data Science Fellowship.

Want to know how our data science learners helped make legal research more accessible and high-tech? Read this blog post to find out.