top of page

Welcome to Health-aware Food Recommendation Scheme

Get into a low carb lifestyle with an easy-to-follow meal plan

home_abstract

Abstract

With the rising incidence of some diseases, such as obesity and diabetes, the healthy diet is arousing increasing attention. However, most existing food-related research efforts focus on recipe retrieval, user preference-based food recommendation, cooking assistance or the nutrition and calorie estimation of dishes, ignoring the personalized health-aware food recommendation. Therefore, in this work, we present a personalized health-aware food recommendation scheme, namely Market2Dish, mapping the ingredients displayed in the market to the healthy dishes eaten at home. The proposed scheme comprises three components, namely recipe retrieval, user health profiling, and health-aware food recommendation. In particular, recipe retrieval aims to recognize the ingredients in the micro-videos taken by the users from the market, and then retrieve recipe candidates from a large-scale recipe dataset. Meanwhile, we propose a deep model to characterize the health conditions of users by capturing the textual health-related information crawled from social networks. To solve the issue that the health-related information is extremely sparse, we incorporate a word-class interaction mechanism into the proposed deep model to learn the fine-grained correlations between the textual tweets and pre-defined health concepts. We ultimately present a novel category-aware hierarchical memory network-based recommender to learn the health-aware user-recipe interactions for better food recommendation. Extensive experiments demonstrate the effectiveness of the health-aware food recommendation scheme. 

Dataset

Dataset

To achieve the personalized health-aware food recommendation based on the available ingredients for users in the market, we constructed three food-related datasets, including the food recommendation dataset, ingredient image dataset, and user health profiling dataset . 

Food Recommendation Dataset

Data collection: 

We crawled a large-scale Chinese recipe dataset from Meishijie, consisting of over 6.6k recipes and each recipe contains rich information, including recipe name, ingredients, cooking instructions, dish pictures, benefits, cooking time, cooking difficulty, etc. Besides, we acquired extensive health diet tips for different people. By integrating the health diet tips and the health tags of Weibo users, we constructed lots of health-aware appropriate and inappropriate dish samples from the crawled recipe dataset for each Weibo user. Ultimately, the food recommendation dataset consists of 64,657 training samples, and each of them includes a Weibo user, the user's health tags, the available ingredients for the user, and about 340 positive recipes, 100 negative recipes for this user from the recipe dataset. 

s3701143_155080680668426.jpg

Dish Name: 

Wonton noodles

Ingredients:

Cabbage 200g; Pork 50g; shrimp meat 50g; noodles ...

instructions: 

1. Cabbage is cleaned, chopped and rubbed repeatedly with hands  ...

Benefits: 

Boosting resistance, Thirst quenching, Diuresis, Appetizer

Cooking skills: 

Cook wonton and noodles in the pot with a little salt, wonton and noodles will be more vigorous.

 

Dish Name: 

Wonton noodles

Ingredients:

Cabbage 200g; Pork 50g; shrimp meat 50g; noodles ...

Instructions: 

1. Cabbage is cleaned, chopped and rubbed repeatedly with hands  ...

Benefits: 

Boosting resistance; appetizer; thirst quenching

Cooking skills: 

Cook wonton and noodles in the pot with a little salt, wonton and noodles will be more vigorous.

 

Ingredient Image Dataset

Ingredient Image Dataset

Data collection

To record the ingredients in the market, we took extensive micro-videos of the ingredients manually. And then we sampled several images from the micro-videos randomly. Ultimately, over 12k images in total were obtained from the video clips. According to our statistics on the ingredient images, there are almost 80 kinds of common ingredients in this dataset, including meat, vegetables, seafood, etc. Thereafter, all sampled images were labeled with the contained ingredients by human. It is worth noting that each image usually contains multiple ingredients and the average number of ingredients per image is 3.61. At last, this dataset is used to recognize the ingredients in the market and then retrieve the available recipe candidates for users. 

User Health Profiling Dataset

43105692_135353644110197_259659702317705
s3-news-tmp-75806-untitled_design_2_1--2

Data collection

Nowadays, social networks have become an essential part in our daily life. Extensive users share their activities, feelings, hobbies there every day, and these user-related information helps us a lot to analyze users' identities, emotions and even the health conditions. To profile the user health, we crawled lots of users' tweets from Weibo, one of the biggest social media platforms in China. And thereafter extensive work for data cleaning and preprocessing was done to improve the quality of the dataset. In addition, we proposed 96 health tags under the guidance of domain experts to cover the common users' identities and health conditions, such as teenager, office worker, obesity, and diabetic patient. Then, we annotated users with the health tags based on their tweets from Weibo using a semi-automatic manually annotation method, incorporating the keyword-based filter rules and human inspection. In particular, each user has a health tag related to his/her age or job at least, and usually has multiple ones. On average, each user has more than 40 tweets crawled from Weibo and 3.89 health tags. Ultimately, we acquired 64,657 Weibo users with 96 kinds of health tags in this dataset. 

Data collection

Nowadays, social networks have become an essential part in our daily life. Extensive users share their activities, feelings, hobbies there every day, and these user-related information helps us a lot to analyze users' identities, emotions and even the health conditions. To profile the user health, we crawled lots of users' tweets from Weibo, one of the biggest social media platforms in China. And thereafter extensive work for data cleaning and preprocessing was done to improve the quality of the dataset. In addition, we proposed 96 health tags under the guidance of domain experts to cover the common users' identities and health conditions, such as teenager, office worker, obesity, and diabetic patient. Then, we annotated users with the health tags based on their tweets from Weibo using a semi-automatic manually annotation method, incorporating the keyword-based filter rules and human inspection. In particular, each user has a health tag related to his/her age or job at least, and usually has multiple ones. On average, each user has more than 40 tweets crawled from Weibo and 3.89 health tags. Ultimately, we acquired 64,657 Weibo users with 96 kinds of health tags in this dataset. 

Case Study

Dish Name: 

Rice with curry beef

Ingredients:

Potato; carrot; onion; chili; beef

Instructions: 

1. Peel the potatoes and carrots and cut them into small pieces. Wash the onion and shred.

2. Wash the peppers and wash them into small pieces and cut the curries into small pieces ...

Benefits: 

Soften blood vessel; reduce blood press; cosmetology

Cooking skills: 

Use the beef stewed in advance to make curry rice, which saves time and tastes good.

home_dataset
Case Study

_Mildred

Materials in market

00001166.jpg
00001357.jpg
00001562.jpg
00001921.jpg
00002141.jpg
00002555.jpg
00003050.jpg
00003345.jpg

02

03

_Mildred

User profiling & Recipe retrieval

User labels:​

  • cold

  • insomnia

  • lose weight

  • dental ulcer

Materials:​

  • tomato

  • cucumber

  • cherry tomato

  • bitter gourd

  • carrot

  • cabbage

  • green bean

  • chili

_Mildred

Dish Recommendation

Hot dishes:​

  • Rice with curry beef

  • Kung pao chicken

  • Vermicelli cabbage

Cold dishes:​

  • Celery with peanut

  • Tomato salad

  • Bitter Melon in Sauce

Soup:​

  • White fungus soup

Dessert:​

  • Egg custard tart

04

FOLLOW ME

Code & Data

framework_jianghao.png
code

01

_Mildred

Weibo content

  • Lose weight punching Day 1st. Breakfast: two fried wheat, a cup of soy milk. Lunch: 25 bowls of small pots. Dinner: a raw tomato. Start exercising at 5 o'clock, so tired and sweaty. (Posted on March 12 at 19:36)

  • Can my mouth ulcers heal? It’s almost two weeks, and even the ulcer medicine has no effect. Every time it is attached, it has the pain of salting the wound. (Posted on March 24 at 10:20)

  • ​​Exhausted [crazy] [crazy]. I have no time to sleep recently. Get up either at five o'clock or at six o'clock every day, and I was still insomnia yesterday [cold]. I am too sleepy! (Posted on April 1 at 19:36)

  • A bad cold for many days, I go to bed after eating medicine almost every day. Hope it will get better soon...... (Posted on April 4 at 19:36)

05

_Mildred

Recipe details

Dish Name: 

Rice with curry beef

Ingredients:

Potato; carrot; onion; chili; beef

Instructions: 

1. Peel the potatoes and carrots and cut them into small pieces. Wash the onion and shred.

2. Wash the peppers and wash them into small pieces and cut the curries into small pieces ...

Benefits: 

Soften blood vessel; reduce blood press; cosmetology

Cooking skills: 

Use the beef stewed in advance to make curry rice, which saves time and tastes good.

405095343.png
bottom of page