Mutual Cognitive Human-Robot Collaborative Manufacturing System 基於人機互認知的機械人協作製造系統

As one of the main realizations of the modern smart factory, human-robot collaborative assembly (HRCA) allows human operators and collaborative robots to perform a variety of manufacturing and assembly tasks in a shared workspace. Due to its flexible automation capabilities, human-machine collaborative assembly has been playing an important role in realizing large-scale personalized production. It combines the high accuracy, reliability and repeatability of collaborative robots with the high flexibility and adaptability of humans to achieve optimized productivity.

The share of collaborative robots in the robotics market has increased dramatically as this collaborative manufacturing model has gained popularity. However, the corresponding application areas and control methods of collaborative robots are still immature, especially when responding to individualized tasks, the robots lack the ability to adapt to new tasks and require engineers to redesign them, a process that increases the time cost and reduces the smoothness of collaboration between the two parties. Mass customization has become a new trend in existing manufacturing models. However, due to the lack of automated solutions for detailed product assembly, manual assembly is still the dominant assembly method used by SMEs for complex customized products.

In order to address the challenges of customized production and lack of robot flexibility, we propose a robotic co-manufacturing system based on human-robot interaction to improve the efficiency of collaboration between both parties. To realize intelligent operations in highly detailed and flexible assembly situations, we develop an augmented reality (AR)-guided human-robot collaborative manufacturing system. It integrates virtual reality task planning and dynamic guidance of visual data, enhances the artificial intelligence of the collaborative robot and AR information sharing, and provides human-supervised control methods in a user-friendly way, thus better integrating the intelligence of the operator and the collaborative robot. It migrates algorithmic models from the field of artificial intelligence to complex industrial scenarios, enabling customized product recognition at the cost of small samples and enhancing the ability of collaborative robots to perform multimodal tasks. The system frees people from repetitive mechanical tasks and makes human-robot collaboration the best choice for industrial organizations.

• Context-aware holistic scene perception and task planning for HRC
• Advanced human-robot collaborative assembly based on behavior understanding**
• Intelligent assembly based on Augmented Reality
• AR dynamic guidance for closed-loop information flow based on Industrial Internet

By deploying the system, we have enhanced the perception, reasoning and cognitive capabilities of the system, including the intelligence of industrial personnel and robots. We achieved the ability to perceive complex environments and track motion paths, to share assembly tasks between humans and collaborative robots based on perceived information, and to adapt to rapidly realize personalized production plans. At the same time, the system realizes interactive visualization and dynamic instruction delivery on an AR platform, resulting in a new model of customized product assembly that is flexible, adaptable, highly automated, and cost-effective.

人機協作裝配作為現代智能工廠的主要實現方式之一，允許人類操作員和協作式機器人在一個共享的工作空間內完成各種製造裝配任務。由於其靈活的自動化能力，人機協作裝配在實現大規模個性化生產方面一直發揮著重要的作用。它將協作機器人高精確度、可靠性及可重複性的特性與人類的高靈活性和適應性相互結合，以實現最優化的生產力。

隨著這種協作製造模式的普及，協作式機器人在機器人市場上所佔的比例明顯增加。然而，相應的應用領域和協作式機器人的控制方法仍未發展成熟，特別是在應對個性化的任務上，機器人缺乏適應新任務的能力，需要工程師對其重新進行程式設計，過程增加了時間成本及降低了雙方協作的流暢性。在現有的製造模式中，大規模定制化生產成為了一種新趨勢。然而，在缺乏產品細節裝配相關自動化解決方案的前提下，人工裝配仍然是中小型企業用於複雜定制化產品的主流裝配方法。

為了解決定制化生產所面對的困難和機器人靈活性不足的問題，我們提出一個基於人機互認知的機械人協作製造系統以改善雙方的協作效率。為了在高度精細和靈活的裝配情況下實現智能操作，我們開發了一套增強現實（AR）引導的人機協作製造系統。它集成了虛擬現實的任務規劃及視覺數據的動態引導，並增強了協作式機器人的人工智能和AR信息共享，以用戶友好的方式提供人為監督的控制方法，從而更好地融合操作員和協作式機器人的智慧。AR輔助的人機協作柔性生產走向了數字化的智能製造，並將人工智能領域的算法模型遷移到複雜的工業場景中，實現了以小樣品為代價的定制化產品的識別，增強協作機器人執行多模式任務的能力。系統將人從重複的機械作業中解放出來，使人機協作成為工業企業的最佳選擇。

通過部署這一系統，我們增強了系統的感知、推理和認知能力，包括工業人員和機器人的智能。實現了對複雜環境的感知和運動路徑的追蹤，根據感知信息在人和協作式機器人之間共享裝配任務，並適應快速實現個性化生產進度。同時，系統在AR平台上交互式地可視化和動態地傳遞指令，形成了一個新的柔性、多用途、快速適應性、高度自動化和成本效益的定制化產品裝配模式。

Students:

• Tsang Chin-lok, Mphil student (Department of Aeronautical and Aviation Engineering)
• Li Chengxi, PhD student (Department of Industrial and Systems Engineering (ISE))
• Kwok Hin-chi, undergraduate student (ISE)
• Other members of RAIDS

Supervisor: Ir Dr Zheng Pai, Assistant Professor (ISE), Wong Tit Shing Young Scholar in Smart Robotics, and leader of RAIDS

Reference/ Awards:

1. APICTA: Being a member of the HKSAR delegation and winning the largest yearly international ICT awards that all the participants are ranked in the top 3 among their regions/ countries. Our team is the first Hong Kong winner in the tertiary student category in the past 4 years.
2. HKICT Grand Award and Gold Award: Ranked in the top 0.3% to win the overall championship among 330 participating teams in Student innovation catorgory
3. Gold Award of Internet+: Ranked in the top 0.009% to win the highest award among 3,400,000 participating teams
4. Second Prize (Entrepreneurship Proposal Track) of The 8th Hong Kong University Student Innovation and Entrepreneurship Competition 2022