Ali Behdarnejad | AI Engineer

Welcome!

Hayyy, I'm Ali and welcome to my website. It's a showcase of my latest projects, work and research. I'm an Artificial Intelligence and Robotic engineer who graduated from Shahid Beheshti University. I would like to work on cutting-edge AI in robotics, NLP, reinforcement learning and areas that have overlap, like using vision language action models(VLAs) with RL fine-tuning. I also would like to work on the Architecture of Deep model in NLP and make them behave similar to the human brain, inspired by cognitive science in robotics and interested in improving exploration and exploitation in reinforcement learning when we worked on P-Explo. In work and research, people say I'm a creative, independent and hard worker. If you would like to chat, email me or send me a message on Telegram at @Ldu2x.

Work experience

AI and robot research engineer at ROIAA lab

As a robotic researcher I got started with DRL and I had to have a overview of the field, I find the best opportunity is from generalist robots which are using LLMs, VLMs, and VLAs so we design a brand new algorithm to address weakness of existing algorithms

Software developer Advance technologys of Sahand

I worked in the company as software developer to write java programs on android OS for POS devises and repairing customer POS with software problems

OpenStack admin and developer at Iranan data suport

As internship, my responsibility in the company was running OpenStack on the server and create VM for the customer and research on NoSQL dataset.

Education

Master of AI and robotic engineering at top rank SBU university

My goul at the strat of this program was to improve my general understanding of AI and get profeational at to one area so I take courses like Image processing, NLP,Mmachine learning, big data and to get profeational at robotics I toke reinfocment learning and mulit agnet systems also my resarch area is using LLM base models in RL and robotics and was able to get working at Robotic lab. some of my master projects are available at my GitHub like Agentic-RAG-with Elasticsearch or Mavc2-Object-detection

Bachelor of computer science at Qom university

In this university I learned a lot about fundamental of computer science like who to program and understand algorithms, compiler,Linux Kernel, network and software engineering

Highlighted Projects

Master thsis: reinfocment fine-tuning Vison language action models(VLAs)

My master tese is about using VLAs and VLMs to make general reward function and policies which is really importiant to make huamnoid robot that can act and like human in this projetc I'm working on state of the art models from MIT and stanford to make a mdoel that mnipulate more complexe taskes for longer time also I'm using human cognitive to make its thinking more like human. soon the paper will be publish in my google scholar
Agentic-RAG-Elasticsearch

As we know the most important part of a good RAG system is retrieval and Elastic search the best open source to use in my openion for multiple reasons the most important is I can have multiple search simultaneously for example always searching of similer embeing is not a good option so I sued embeding with Lexical similarits after this accuracy of the RAG model has improved by 145 present which is also improvable and code is avalbel on my github.
Mavc2-Object-detection and computer vision projects

in this project I used classic computer vision to denoise and CNNs to identify and detect some objects in webots simulator. andsome other projects for 3 months with a team. my first interst when I start my master was computer vision and I stil love it for wheil I implement CD-Unet for segmentation best project that I worked on it was Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models after persenting and testing this complex and multi step project I thouthe it move me form zero to hero for a ferst term master student code of thoes are avalbel at my github.

Publication

P-Explo: Periodic Exploration before Convergence in Deep Reinforcement Learning

Abstract— Action selection strategy is an important part of any reinforcement learning algorithm, because it decided where to use learned policy instead of exploring the environment based on random action. If we have best policy with a bad action selection strategy, the agent will perform not as good as it can be. Traditionally, action selection mechanisms apply exploration-exploitation trade-off by uniformly decaying the exploration factor. However, in real word we usually return to perform exploration, periodically. This idea encourages us to test an exponential periodic decay for -greedy and Boltzmann strategy and compare it with an exponential decay. We apply this idea on Dueling Double Deep Q-Network(D3QN) with soft target network update, Dueling Deep Q-Network(D2QN) and Deep Q-Network (DQN). We test two main environment settings with and without disturbances to study the robustness of the model against perturbances. Based on the performed experiments the proposed method’s success rate is 8 times greater than the traditional decaying approach. Finally, the implementation code is available at https://github.com/AliBehdar/sindecay.git. The paper is sent to IICAI2026 and get accepted so soon available