Hello, I'm Kulbir Singh Ahluwalia

Active Semantic Mapping with Mobile Manipulator in Horticultural Environments

We introduce an efficient active semantic mapping approach for horticultural robotics, using a mobile manipulator with an RGB-D camera. Probabilistic semantic octomaps are used to detect target regions of interest such as fruits, generate candidate viewpoints, and compute information gain for next-best-view planning. An efficient ray-casting strategy and a novel information gain function accounting for semantics and occlusions is introduced for efficient target-focused map exploration.

Plant Placement using Natural Language Grounding

This poster presents a text-enabled FarmBot system that enables users to control a robotic gardening FarmBot system via natural language. Using a custom Python wrapper built on the FarmBot REST API, natural language commands are grounded using real-time robot state and translated into executable code with a fine-tuned CodeT5 model. The system generates valid plant placement configurations that satisfy natural language defined spatial constraints.

DeepPaSTL: Spatio-Temporal Deep Learning Methods for Predicting Long-Term Pasture Terrains Using Synthetic Datasets

DeepPaSTL aims to accurately forecast long-term pasture growth, tackling the challenge of estimating pasture biomass without relying on extensive site-specific data or frequent field measurements. This approach enables predicting pasture evolution without monitoring fields regularly, using past observed pasture heights as input. DeepPaSTL introduces a bi-directional ConvLSTM encoder–decoder to learn the spatio-temporal pasture growth dynamics purely from spatial height measurements.

Intermittent Deployment for Large-Scale Multi-Robot Forage Perception: Data Synthesis, Prediction, and Planning

Targets large-scale pasture monitoring for precision agriculture, deploying a team of robots to track grassland growth for optimal rotational grazing and land productivity, addressing the lack of timely growth data in current practice. Proposes an integrated pipeline combining synthetic data generation, deep neural network-based spatiotemporal prediction, and an intermittent multi-robot deployment strategy to periodically survey evolving pastureland at low cost.​

Active Semantic Mapping with Mobile Manipulator in Horticultural Environments

An abstract accepted at the 40th Anniversary of the IEEE Conference on Robotics and Automation (ICRA@40), 2024.

Smartphone Optical Sensors

An article featuring the multispectral Fundus Eye camera prototype, as presented in Optics and Photonics News.

Reinforcement Learning Integrated with Supervised Learning for Training of Near Infrared Spectrum Data for Non-Destructive Testing of Fruits

A conference presentation on combining reinforcement and supervised learning for non-destructive testing of fruits using near infrared spectrum data.

Teaching Assistant - CS444: Deep Learning for Computer Vision

Instructor: Dr. Svetlana Lazebnik
• Updated and verified starter code for assignments, and answered student questions during office hours and through Campuswire.
• Assessed student submissions via SpeedGrader on Canvas, and designed multimodal quiz questions, including single- choice, multiple-choice, and matching formats.

Enhancing Stereo Depth Maps through RGBD-Conditioned Generative Models

Active Semantic Mapping with Mobile Manipulator for Precision Agriculture

Turning Zero Shot into Few Shot via Self-prompting

Implemented Linear, Logistic, Polynomial regression, SVM, Convolutional, and Attention based models

VAE and GAN implementation

Implemented VAE & GAN for digit generation task.

RESNet implementation

Implemented RESNet for classification task on MNIST dataset.

Neural transition-based dependency parser

Machine translation with RNN and Transformers

Text classification with CNN and RNN

SLAM from 2D LiDAR data using split and merge line extraction algorithm

State estimation using Extended Kalman Filter for GPS+IMU+Encoder sensor fusion

Processed data from RTK-GPS, IMU and encoders to plot trajectory of a field robot

Autonomous Vaccine Delivery Robot

Designed an autonomous robot capable of navigating and localizing itself in a test arena using QR codes and arrows. It uses a RGB camera, IMU, optical encoders, and an ultrasonic sensor to detect, retrieve and transport user-specified blocks. Featured video, Robot videos, Featured post

Image segmentation using superpixels

Built a segmentation network using SLIC superpixels as input. A pretrained VGG16 network had its last layers replaced by fully connected layers to classify superpixels. (Accuracy 98%)

Persistent-Monitoring using Multi-Robot (UAV-UGV) Coordination

Optimized a GestureGAN for resource constrained settings

Used MobileNet to optimize cross-view image generation with a 5.7X reduction in parameters.

Self-adjusting roadmaps

Navigation in unknown environments using LD-PRM.

Estimated the motion of a car using Visual odometry

Color segmentation using Gaussian mixture models & Expectation maximization

Image Classification using CNN

AR-Tag detection

Superimposed an image and virtual cube on an AR tag.

Tracked moving objects using Lucas-Kanade Tracker

Baxter transporting cubes in Gazebo

Simulated a Baxter robot transporting cubes between tables in Gazebo using ROS Kinetic. Waypoints were generated using Rviz and obstacles were avoided in the custom-designed Gazebo world.

Implemented A star algorithm for Path Planning on Turtlebot 3

Implemented the A* algorithm in a configuration space with obstacles. The Turtlebot 3 obeys non-holonomic constraints with 8 combinations of two user-defined RPMs.

Path planning for point and rigid robot using Djikstra's Algorithm

Lane detection and Turn prediction for self driving car

Developed an algorithm using hough transform and histogram of lanes. Also implemented homography and warp perspective functions from scratch for overlays.

Agile Robotics for Industrial Automation Competition (ARIAC) 2019

Developed an industrial system with UR 10 robotic arms, conveyor belts, and AGVs. The system picked parts from a conveyor, disposed faulty items, assembled orders, and delivered them using AGVs.

Designed a PID controller for Turtlebot 3

Modelled a UR 5 arm with Parallel Gripper in Rviz

Simulated a 7 DOF UR5 arm using Moveit and Rviz. Calculated DH parameters and computed forward kinematics manually, verified via Peter Corke’s Robotics toolbox. Simulation videos

Designed a LQR and LQG controller

Developed controllers for two inverted pendulums on a moving cart.

Teleoperated gesture controlled robotic arm

Engineered a prototype to transport objects between rooms via web-based remote access with live video and gesture control. The robot featured on-board power, a custom LED light source for low-light navigation, smart device control, and a speaker for prerecorded messages.

Pick n place transporter bot

Awarded First Prize in IIT Roorkee and placed 6th out of 400 teams at IIT Bombay. Video 1 Video 2

Smart Garden

Won First Prize in the Texas Instruments Hardware Hackathon.

First prize in B.Tech. Major Project, 2019

First prize in final year MAJOR PROJECT in the B.Tech. Examination of Electrical Engineering, 2015-19 titled Teleoperated Gesture controlled Robotic arm.

Certificate of Appreciation, (2017,2018)

Received certificate of appreciation for contributions to IEEE PEC (2017,2018).

Programming in Python, 2019

Data Structures & Algorithms in Python, 2022

National Bal Shree Award, 2011

Awarded with the National Bal Shree Award in Creative Scientific Innovations by the Ministry of Human Resource Development, Govt. of India. It consisted of a series of scientific hands-on tests and interviews at city, zonal and national level.