3D Diffusion Models for Generating and Understanding 3D Scenes. Diffusion models, such as DALL-E2 and Imagen, have achieved remarkable success in generating photorealistic images and hold promise to solve long-standing computer vision problems. However, 3D scene generation remains unexplored. This research project aims to bridge the gap by developing 3D diffusion models capable of generating complete 3D scenes. This will advance our theoretical understanding of diffusion in complex 3D environmen ....3D Diffusion Models for Generating and Understanding 3D Scenes. Diffusion models, such as DALL-E2 and Imagen, have achieved remarkable success in generating photorealistic images and hold promise to solve long-standing computer vision problems. However, 3D scene generation remains unexplored. This research project aims to bridge the gap by developing 3D diffusion models capable of generating complete 3D scenes. This will advance our theoretical understanding of diffusion in complex 3D environments and open up new possibilities for applications in fields such as virtual reality, architecture, and city planning. The proposed 3D diffusion models will also enhance the accuracy of computer vision tasks related to 3D scene understanding, such as object detection, tracking, and semantic segmentation.Read moreRead less
Discovery Early Career Researcher Award - Grant ID: DE240100967
Funder
Australian Research Council
Funding Amount
$366,000.00
Summary
Open-world computer vision by detecting and tracking hierarchical objects. This project examines the problem of detecting and tracking objects using computer vision. A fundamental limitation of current algorithms is that they require labelled training data for every object class and therefore cannot be trusted to operate in unconstrained environments. This project aims to address this limitation using novel techniques that incorporate hierarchical relationships between object classes. Expected o ....Open-world computer vision by detecting and tracking hierarchical objects. This project examines the problem of detecting and tracking objects using computer vision. A fundamental limitation of current algorithms is that they require labelled training data for every object class and therefore cannot be trusted to operate in unconstrained environments. This project aims to address this limitation using novel techniques that incorporate hierarchical relationships between object classes. Expected outcomes include new paradigms for algorithm design and evaluation, and establishing the problem as a focus of international research. The key practical benefit would be to accelerate the wider deployment of visual perception in applications such as autonomous vehicles, interactive robotics, and video analysis.Read moreRead less
Learning to Reason in Reinforcement Learning. Deep Reinforcement Learning (RL) uses deep neural networks to represent and learn optimal decision-making policies for intelligent agents in complex environments. However, most RL approaches require millions of episodes to converge to good policies, making it difficult for RL to be applied in real-world scenarios taking significant resources. This project aims to equip RL with capabilities such as counterfactual reasoning and outcome anticipation to ....Learning to Reason in Reinforcement Learning. Deep Reinforcement Learning (RL) uses deep neural networks to represent and learn optimal decision-making policies for intelligent agents in complex environments. However, most RL approaches require millions of episodes to converge to good policies, making it difficult for RL to be applied in real-world scenarios taking significant resources. This project aims to equip RL with capabilities such as counterfactual reasoning and outcome anticipation to significantly reduce the number of interactions required, improve generalisation, and provide the agent with the capability to consider the cause-effects. These improvements would narrow the gap between AI and human capabilities and broaden the adoption of RL in real-world applications.Read moreRead less
Visual methods for advanced automation of underwater manipulation. This project will increase the autonomy of underwater robotic systems engaged in intervention and inspection tasks. Such activities are essential for the operation of subsea robotic systems used in offshore industries, scientific exploration and defence. Our approach will improve perception and situational awareness through the principled fusion of multiple navigation and camera sensors. We will use this improved scene understand ....Visual methods for advanced automation of underwater manipulation. This project will increase the autonomy of underwater robotic systems engaged in intervention and inspection tasks. Such activities are essential for the operation of subsea robotic systems used in offshore industries, scientific exploration and defence. Our approach will improve perception and situational awareness through the principled fusion of multiple navigation and camera sensors. We will use this improved scene understanding to effectively plan the motion of vehicles and manipulators through larger and more complex workspaces, enabling semi-supervised and autonomous task execution. Our project will demonstrate these capabilities in real-world deployments relevant to industry and marine science.Read moreRead less
Deep Adder Networks on Edge Devices. This project aims to empower edge devices with intelligence by developing advanced deep neural networks that address the conflict between the high resource requirements of deep learning and the generally inadequate performance of the edge. Multiplication has been the dominant type of operation in deep learning, though the addition is known to be much cheaper. This project expects to yield theories and algorithms that allow deep neural networks consisting of n ....Deep Adder Networks on Edge Devices. This project aims to empower edge devices with intelligence by developing advanced deep neural networks that address the conflict between the high resource requirements of deep learning and the generally inadequate performance of the edge. Multiplication has been the dominant type of operation in deep learning, though the addition is known to be much cheaper. This project expects to yield theories and algorithms that allow deep neural networks consisting of nearly pure additions to fulfil the requisites of accuracy, robustness, calibration and generalisation in real-world computer vision tasks. The success of this project will benefit deep learning-based products on smartphones or robots in health and cybersecurity.Read moreRead less
Discovery Early Career Researcher Award - Grant ID: DE230101591
Funder
Australian Research Council
Funding Amount
$419,154.00
Summary
Towards Real-world Continual Learning on Unrestricted Task Steams. This project aims to enable machines to continually learn without forgetting and accumulate knowledge from the sequential data streams containing diverse tasks. This project expects to advance the continual learning to unrestricted real-world task steams that are long-term and complex and promote artificial intelligence toward the human-level intelligence that can automatically evolve during interaction with the world. Expected o ....Towards Real-world Continual Learning on Unrestricted Task Steams. This project aims to enable machines to continually learn without forgetting and accumulate knowledge from the sequential data streams containing diverse tasks. This project expects to advance the continual learning to unrestricted real-world task steams that are long-term and complex and promote artificial intelligence toward the human-level intelligence that can automatically evolve during interaction with the world. Expected outcomes of this project include the paradigm-shifting continual learning framework and techniques for handling unrestricted task steams in real-world scenarios. They will benefit society and the economy nationally and internationally by enhancing the applicability of artificial intelligence.Read moreRead less
Generative Visual Pre-training on Unlabelled Big Data. This project aims to develop a generative visual pre-training of large-scale deep neural networks on unlabelled big data. Developing pre-trained visual models that are accurate, robust, and efficient for downstream tasks is a keystone of modern computer vision, but it poses challenges and knowledge gaps to existing unsupervised representation learning. Expected outcomes include new theories and algorithms for unsupervised visual pre-training ....Generative Visual Pre-training on Unlabelled Big Data. This project aims to develop a generative visual pre-training of large-scale deep neural networks on unlabelled big data. Developing pre-trained visual models that are accurate, robust, and efficient for downstream tasks is a keystone of modern computer vision, but it poses challenges and knowledge gaps to existing unsupervised representation learning. Expected outcomes include new theories and algorithms for unsupervised visual pre-training, which are anticipated to deepen our understanding of visual representation and make it easier to build and deploy computer vision applications and services. Examples of benefits include modernising machines in manufacturing and farming with visual intelligence. Read moreRead less
Home helper robots: Understanding our future lives with human-like AI. This fellowship aims to understand and plan for the social effects of embedding ‘cute’ home helper robots into people’s everyday lives. The project is expected to generate new knowledge and resources to understand and respond to the emerging opportunities and risks associated with home helper robots, including their ability to support household tasks, and to provide child and aged care and companionship. Expected outcomes inc ....Home helper robots: Understanding our future lives with human-like AI. This fellowship aims to understand and plan for the social effects of embedding ‘cute’ home helper robots into people’s everyday lives. The project is expected to generate new knowledge and resources to understand and respond to the emerging opportunities and risks associated with home helper robots, including their ability to support household tasks, and to provide child and aged care and companionship. Expected outcomes include an improved understanding of anthropomorphised robots in everyday life and innovation in home helper robot theory and imaginaries. This should provide benefits such as informing robot design and policy to improve social outcomes, consumer protections and human-robot relationships.Read moreRead less
Embracing Changes for Responsive Video-sharing Services. Video-sharing platforms are a critical information channel for the public. Increasing scale and shifts in user base, with Generation Z now as the dominant user, have resulted in an unprecedented amount of ubiquitous changes in the content and users of these platforms which greatly challenges the responsiveness and quality of the services provided. This project aims to design innovative algorithms to effectively predict and leverage changes ....Embracing Changes for Responsive Video-sharing Services. Video-sharing platforms are a critical information channel for the public. Increasing scale and shifts in user base, with Generation Z now as the dominant user, have resulted in an unprecedented amount of ubiquitous changes in the content and users of these platforms which greatly challenges the responsiveness and quality of the services provided. This project aims to design innovative algorithms to effectively predict and leverage changes, optimise the value of changes, and extract insights from changes for diverse downstream applications of video-sharing platforms. The expected outcomes will create new-generation representation learning techniques, and provide practical tools to amplify the socioeconomic values of video-sharing platforms.Read moreRead less
Discovery Early Career Researcher Award - Grant ID: DE230101567
Funder
Australian Research Council
Funding Amount
$453,054.00
Summary
Listening to Nature: Transforming Bioacoustics through Spatial Audio. This project aims to research new 3D spatial audio processing techniques to analyse natural sounds for environmental conservation, while meeting the tasks, demands and data characteristics inherent to bioacoustics. Expected outcomes include new, accurate and efficient bioacoustics computation technologies, generalisable across different terrestrial regions, species types and environment changes. These could dramatically enhanc ....Listening to Nature: Transforming Bioacoustics through Spatial Audio. This project aims to research new 3D spatial audio processing techniques to analyse natural sounds for environmental conservation, while meeting the tasks, demands and data characteristics inherent to bioacoustics. Expected outcomes include new, accurate and efficient bioacoustics computation technologies, generalisable across different terrestrial regions, species types and environment changes. These could dramatically enhance the efficacy of current bioacoustic monitoring systems while opening up new research directions. Resulting technology could be adopted for immediate tasks like the monitoring of bushfire recovery efforts, and more generally, for the management and conservation of Australian natural resources.Read moreRead less