Research
My work sits at the intersection of hardware design and AI security — building efficient accelerators for deep learning models and hardening them against adversarial threats.
Hardware-Software Co-Design for Efficient ViT Inference
Vision Transformers (ViTs) achieve state-of-the-art accuracy but are computationally expensive. This research addresses the challenge of deploying ViTs efficiently on dedicated hardware through informative patch/token selection and custom accelerator design.
- Proposed lightweight, image-adaptive informative patch selection techniques for efficient yet accurate ViT acceleration.
- Designed and implemented a dedicated ASIC (HAVIT) to accelerate both patch selection and the full ViT pipeline.
- Analyzed token selector placement in the ViT architecture, optimizing the accuracy–computation trade-off in attention-based methods.
- Proposed a feature-based token pruning strategy for early layers, reducing reliance on attention scores.
- Developed a hybrid multi-stage token pruning methodology, reducing execution time by 29–40% with minimal accuracy loss.
- Implemented hardware accelerator for ViT backbone on FPGA using Vitis HLS and Vivado.
Adversarial Robustness of Vision Transformers
Adversarial attacks pose a serious threat to the real-world deployment of deep learning models. This line of research develops both attack methods and defenses specifically targeting ViTs, with a focus on scenarios where models are optimized for efficient inference on custom hardware.
- Developed model-agnostic defenses for adversarial patch attacks on Vision Transformers (STRAP-ViT).
- Analyzed efficiency-targeted adversarial attacks that exploit input-adaptive optimization strategies in ViTs.
- Proposed approaches to enhance robustness of optimized/accelerated ViT inference pipelines.
- Investigating stronger energy-based attacks on efficient AI inference hardware.
- Working on improving robustness of large language models against jailbreak prompts.
Secure Hardware Accelerator for Resilience Against Adversarial Attacks
This project bridges hardware design and AI security by building a dedicated FPGA-based accelerator that can detect and neutralize adversarial noise in real time, enabling secure ViT deployment in resource-constrained edge settings.
- Developing a lightweight rectifier algorithm to eliminate adversarial noise from ViT inputs.
- Implementing a custom hardware accelerator on FPGA for real-time rectification.
- Evaluating system performance under white-box and black-box attack scenarios.
Reinforcement Learning-Based Cache Prefetcher Design
Cache prefetching is critical for memory system performance. This project improves RL-based prefetchers by refining the Q-value update process to increase cache coverage while reducing pollution, and estimates hardware overhead through custom implementations.
- Modified existing RL-based Q-value update process (multi-step SARSA) to improve cache coverage and reduce cache pollution.
- Developed hardware to estimate area and power overhead of the proposed prefetcher.
Efficient Disease Prediction Using Vision AI on Blood Samples
An end-to-end application project combining fine-tuned ViT models for blood sample analysis with a custom FPGA hardware accelerator for deployment.
- Fine-tuned a Vision Transformer on blood sample images for accurate disease detection.
- Developed a custom hardware accelerator to deploy the algorithm on an FPGA.