Abstract

Robotic pick-and-place has been researched for a long time to cope with uncertainty of novel objects and changeable environments. Past works mainly focus on learning-based methods to achieve high precision. However, they have difficulty being generalized for the limitation of specified training models. To break through this drawback of learning-based approaches, we introduce a new perspective of similarity matching between novel objects and a known database based on category-association to achieve pick-and-place tasks with high accuracy and stabilization. We calculate the category name similarity using word embedding to quantify the semantic similarity between the categories of known models and the target real-world objects. With a similar model identified by a similarity prediction function, we preplan a series of robust grasps and imitate them to plan new grasps on the real-world target object. We also propose a distance-based method to infer the in-hand posture of objects and adjust small rotations to achieve stable placements under uncertainty. Through a real-world robotic pick-and-place experiment with a dozen of in-category and out-of-category novel objects, our method achieved an average success rate of 90.6% and 75.9% respectively, validating the capacity of generalization to diverse objects.

Movies

Citation

@article{Sakuma2023,
     author = {Chen, Hao and Kiyokawa, Takuya and Weiwei, Wan and Harada, Kensuke},
     title = {Category-Association Based Similarity Matching for Novel Object Pick-and-Place Task},
     journal = {IEEE Robotics and Automation Letters (RA-L)},
     url = {https://ieeexplore.ieee.org/document/9691866},
     volume = {7},
     number = {2},
     pages = {2961--2968},
     year = {2022},
     month = {April},
     publisher = {IEEE},
     doi = {10.1109/LRA.2022.3145956}
 }

Description

This research project is taking place in Harada Laboratory, Osaka University, Japan. This research is subsidized by New Energy and Industrial Technology Development Organization (NEDO) under a project JPNP20016. This paper is one of the achievements of joint research with ROBOT Industrial Basic Technology Collaborative Innovation Partnership (ROBOCIP).

Contact

We welcome collaboration.
For any questions, clarifications, please get in touch with: Takuya Kiyokawa
Please send an email from your institutional mail address, and do include your affiliation details.

Abstract

Robotic pick-and-place has been researched for a long time to cope with uncertainty of novel objects and changeable environments. Past works mainly focus on learning-based methods to achieve high precision. However, they have difficulty being generalized for the limitation of specified training models. To break through this drawback of learning-based approaches, we introduce a new perspective of similarity matching between novel objects and a known database based on category-association to achieve pick-and-place tasks with high accuracy and stabilization. We calculate the category name similarity using word embedding to quantify the semantic similarity between the categories of known models and the target real-world objects. With a similar model identified by a similarity prediction function, we preplan a series of robust grasps and imitate them to plan new grasps on the real-world target object. We also propose a distance-based method to infer the in-hand posture of objects and adjust small rotations to achieve stable placements under uncertainty. Through a real-world robotic pick-and-place experiment with a dozen of in-category and out-of-category novel objects, our method achieved an average success rate of 90.6% and 75.9% respectively, validating the capacity of generalization to diverse objects.

Movies

Citation

@article{Chen2022,
     author = {Chen, Hao and Kiyokawa, Takuya and Weiwei, Wan and Harada, Kensuke},
     title = {Category-Association Based Similarity Matching for Novel Object Pick-and-Place Task},
     journal = {IEEE Robotics and Automation Letters (RA-L)},
     url = {https://ieeexplore.ieee.org/document/9691866},
     volume = {7},
     number = {2},
     pages = {2961--2968},
     year = {2022},
     month = {April},
     publisher = {IEEE},
     doi = {10.1109/LRA.2022.3145956}
 }

Description

This research project is taking place in Harada Laboratory, Osaka University, Japan. This research is subsidized by New Energy and Industrial Technology Development Organization (NEDO) under a project JPNP20016. This paper is one of the achievements of joint research with ROBOT Industrial Basic Technology Collaborative Innovation Partnership (ROBOCIP).

Contact

We welcome collaboration.
For any questions, clarifications, please get in touch with: Takuya Kiyokawa
Please send an email from your institutional mail address, and do include your affiliation details.

Abstract

Owing to human labor shortages, the automation of labor-intensive manual waste-sorting is needed. The goal of automating the waste-sorting is to replace the human role of robust detection and agile manipulation of the waste items by robots. To achieve this, we propose three methods. First, we propose a combined manipulation method using graspless push-and-drop and pick-and-release manipulation. Second, we propose a robotic system that can automatically collect object images to quickly train a deep neural network model. Third, we propose the method to mitigate the differences in the appearance of target objects from two scenes: one for the dataset collection and the other for waste sorting in a recycling factory. If differences exist, the performance of a trained waste detector could be decreased. We address differences in illumination and background by applying object scaling, histogram matching with histogram equalization, and background synthesis to the source target-object images. Via experiments in an indoor experimental workplace for waste-sorting, we confirmed the proposed methods enable quickly collecting the training image sets for three classes of waste items, i.e., aluminum can, glass bottle, and plastic bottle and detecting them with higher performance than the methods that do not consider the differences. We also confirmed that the proposed method enables the robot quickly manipulate them.

Movies

Citation

@article{Kiyokawa2021sorter,
     author = {Takuya Kiyokawa and Hiroki Katayama and Yuya Tatsuta and Jun Takamatsu and Tsukasa Ogasawara},
     title = {Robotic Waste Sorter with Agile Manipulation and Quickly Trainable Detector},
     journal = {IEEE ACCESS},
     volume = {9},
     number = {},
     pages = {124616--124631},
     year = {2021},
     month = {September},
     day = {},
     publisher = {},
     doi = {10.1109/ACCESS.2021.3110795}
 }

Description

This research project is taking place in Robotics Laboratory, NAIST, Japan. This work was supported in part by the projects, JPNP14004 and JPNP20012, subsidized by the New Energy and Industrial Technology Development Organization (NEDO).

Contact

For any questions, clarifications, please get in touch with: Takuya Kiyokawa
Please send an email from your institutional mail address, and do include your affiliation details.

Abstract

Aiming to generate easy-to-handle assembly sequences for robotic assembly, this study tackles assembly sequence generation by considering two tradeoff objectives: (1) insertion conditions and (2) degrees of constraints among assembled parts. We propose a multiobjective genetic algorithm to balance these two objectives for generating assembly sequences. Furthermore, the method of extracting part relation matrices including interference-free, insertion, and degree of constraint matrices is extended for application to 3D computer-aided design (CAD) models, including deformable parts. The interference of deformable parts with other parts can be easily investigated by scaling parts. A simulation experiment was conducted using the proposed method, and the results show the possibility of obtaining Pareto-optimal solutions of assembly sequences for a 3D CAD model with 33 parts including a deformable part. This approach can potentially be extended to handle various types of deformable parts and to explore graspable sequences during assembly operations.

Movies

Citation

@inproceedings{Kiyokawa2020seqgen,
     author = {Takuya Kiyokawa and Jun Takamatsu and Tsukasa Ogasawara},
     title = {Assembly Sequences Based on Multiple Criteria Against Products with Deformable Parts},
     booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
     volume = {},
     number = {},
     pages = {975--981},
     year = {2021},
     doi = {10.1109/ICRA48506.2021.9561925}
 }

Description

This research project is taking place in Robotics Laboratory, NAIST, Japan.

Contact

We welcome collaboration.
For any questions, clarifications, please get in touch with: Takuya Kiyokawa
Please send an email from your institutional mail address, and do include your affiliation details.

Abstract

To design a general-purpose assembly robot system that can handle objects of various shapes, we propose a soft jig capable of deforming according to the shape of assembly parts. The soft jig is based on a jamming gripper used for robot manipulation as a general-purpose robotic gripper developed in the field of soft robotics. The soft jig has a flexible membrane made of silicone, which has a high friction, elongation, and contraction rate for keeping parts strictly fixed. The inside of the membrane is filled with glass beads to achieve a jamming transition. The usability of the soft jig was evaluated from the viewpoint of the versatility and fixing performance for various shapes and postures of parts in assembly operations.

Movies

Citation

@inproceedings{Kiyokawa2020softjig,
     author = {Takuya Kiyokawa and Tatsuya Sakuma and Jun Takamatsu and Tsukasa Ogasawara},
     title = {Soft-Jig-Driven Assembly Operations},
     booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
     volume = {},
     number = {},
     pages = {3466--3472},
     year = {2021},
     doi = {10.1109/ICRA48506.2021.9562008}
 }

Description

This research project is taking place in Robotics Laboratory, NAIST, Japan.

Contact

We welcome collaboration.
For any questions, clarifications, please get in touch with: Takuya Kiyokawa
Please send an email from your institutional mail address, and do include your affiliation details.

Abstract

Automated factories use deep-learning-based vision systems to accurately detect various products. However, training such vision systems requires manual annotation of a significant amount of data to optimize the large number of parameters of the deep convolutional neural networks. Such manual annotation is very time-consuming and laborious. To reduce this burden, we propose a fully automated annotation approach without any manual intervention. To do this, we associate one visual marker with one object and capture them in the same image. However, if an image showing the marker is used for training, normally, the neural network learns the marker as a feature of the object. By hiding the marker with a noise mask, we succeeded in reducing this erroneous learning. Experiments verified the effectiveness of the proposed method in comparison with manual annotation, in terms of both the time needed to collect training data and the resulting detection accuracy of the vision system. The time required for data collection was reduced from 16.1 to 1.87 h. The accuracy of the vision system trained with the proposed method was 87.3%, which is higher than the accuracy of a vision system trained with the manual method.

Movies

Citation

@article{Kiyokawa2019ral,
     author = {Takuya Kiyokawa and Keita Tomochika and Jun Takamatsu and Tsukasa Ogasawara},
     title = {Fully Automated Annotation With Noise-Masked Visual Markers for Deep-Learning-Based Object Detection},
     journal = {IEEE Robotics and Automation Letters (RA-L)},
     url = {https://ieeexplore.ieee.org/document/8641376},
     volume = {4},
     number = {2},
     pages = {1972--1977},
     year = {2019},
     month = {February},
     publisher = {IEEE},
     doi = {10.1109/LRA.2019.2899153}
 }

Description

This research was partially supported by Exploratory IT Human Resources Project (MITOU Program) of Information-technology Promotion Agency, Japan (IPA) in the fiscal year 2018.
We thank DENSO WAVE INCORPORATED for lending us COBOTTA® (en/ja) robot and technical supports for the robot. SAKURA internet Inc. (en/ja) is providing us the High-powered computing resource (ja). We highly appreciate the generous kindness. We would also like to show our gratitude to Enplas Corporation for providing us High-precision visual markers.

Contact

We are looking for someone to drive this research into a product or business and also welcoming collaborative researchers.
For any questions, clarifications, please get in touch with: Takuya Kiyokawa
Please send an email from your institutional mail address, and do include your affiliation details.