What’s beneath the clouds

For her PhD, Stephanie was inspired by the tricorder from the Star Trek TV series and movies, which is a handheld medical device. Stephanie’s aim was to build a non-invasive device that measures vital signs in the human body, such as blood pressure and respiratory rate.

However, the COVID-19 pandemic placed several restraints on her research project. With social distancing in place at the time, Stephanie decided to use data from an American Open Access database.

“We analysed publicly available data from photoplethysmogram sensors, which are light-based sensors like those in-built in regular fitness watches. We also drew upon data from electrocardiogram (ECG) sensors,” Stephanie says, adding that this data is usually collected in-person within research labs, where participants often have stickers attached to their chest.

“There had been some research in the area already, but my method was performing better than a lot of what we had seen before, so that was great,” Stephanie says. “We also looked at using this data, which we call ‘vital sign data’, to predict severity of illness.

“This actually became more relevant during the pandemic, as we wanted to identify when someone needed the highest level of treatment,” Stephanie says. “We did mortality risk prediction using these vital signs for both an adult and a neonatal (newborn) cohort. And that worked really well.”

Since finishing her PhD in 2021, Stephanie has remained true to her passion for health care, albeit with one exception. After hearing about a sea surface monitoring research project from a former fellow JCU student, who was working for Geoscience Australia, Stephanie decided that this would be a welcome challenge that she couldn’t resist.

“Applying AI to a new type of data isn't a huge step once you have a grasp on how AI systems work,” Stephanie says. Her friend had introduced her to Zhi Huang, a marine environmental modeller from Geoscience Australia, who was looking to fill gaps in satellite images.

When satellites take snapshots of the ocean, there are always at least some areas covered by clouds, and the current artificial intelligence-based models have not been accurate in terms of calculating what is beneath these clouds.

The research was a team effort, Stephanie explains. “Zhi Huang from Geoscience Australia, who invited me to help out, first explained the actual geoscience behind it to me. Taking that further and applying it with AI was reasonably easy,” Stephanie says.

“Zhi was also the one who collected the data from the satellite and did some initial pre-processing on it as well before he passed the data on to me. I was responsible for the development of the AI and a lot of the processing of the data.”

JCU’s Bronson Philippa, Senior Lecturer for Electronic Systems and IoT Engineering, was also on the team. “Bronson helped me with brainstorming ideas, including a novel time-based penalty that our final AI model used to help it fill gaps more accurately,” she says.

A new way of looking at sea surface data

Stephanie, Zhi and Bronson weren’t the first to conduct this kind of research, but the models used previously needed a lot of computational power and weren’t very accurate. “We were trying to obtain a lower error rate in terms of temperatures and structure of the currents,” Stephanie says.

The team experimented with several AI models for almost six months, and there were many disappointments. “Some conventional machine learning approaches, such as convolutional neural networks for image-based data, weren’t successful.

"These models are good at finding features within images, but we had no luck with those, as they weren’t accurate,” Stephanie says. “We also tried out more advanced models that are usually used for reconstruction tasks. They didn’t produce the results we were looking for either.”

In the end, however, Stephanie, Zhi and Bronson were surprised that the model that did work was both fast and light, at least in terms of the required computer hardware. The final model could be readily run on a conventional laptop.

“We decided to use a bidirectional long-short term memory structure (BiLSTM), which is useful in machine learning,” Stephanie says. “This model is good at remembering data that the AI has seen elsewhere in a sequence. It produced better results than previous models and in a shorter time.”

So, the hard work for Stephanie, Zhi and Bronson has paid off, but also for future researchers, as all of the code and data associated with this project have been made Open Access on Stephanie’s GitHub page for other researchers to use. This is especially helpful when monitoring changes in ocean temperatures and ocean currents over time.

Returning to AI-based health research

After the detour into sea surface research, Stephanie is now working on a new healthcare project with Professor Yogavijayan Kandasamy from JCU’s Australian Institute of Tropical Health and Medicine (AITHM). Stephanie and Yogavijayan are currently looking to develop a novel system that measures blood pressure and other vital signs in premature babies in a non-invasive way.

“We have a little bit of grant funding, and we are slowly working through the very long ethics approval process at the moment,” Stephanie says. “Ultimately, we are hoping to bring a PhD student on board to support this research.”

Students interested in working on this PhD project, or who are looking for PhD supervision in a related field, are invited to contact Stephanie via her JCU researcher page.