Real-Time Implementation of Stable Diffusion on Dual-Core Cadence Tensilica DSP for a Tier 1 Semiconductor OEM
AI-Powered Text-to-Image Generation
Client
Tier 1 Semiconductor OEM
Industry
Semiconductors
Edge Computing
Service
Generative AI Optimization
DSP Model Deployment
Use Case
On-Device Image Generation
DSP-Based AI Inference
Technology
Stable Diffusion
Cadence Tensilica Vision Q7 DSP
Goal
Develop and deploy a Stable Diffusion-based text-to-image generation pipeline on a dual Cadence Tensilica Vision Q7 DSP architecture for a Tier 1 Semiconductor OEM, enabling Generative AI inference on DSP-centric embedded hardware without relying on GPUs or dedicated NPUs. The objective was to optimize the complete diffusion pipeline to operate efficiently within the compute, memory, and bandwidth constraints of the target platform.
Problem Statement
The client required a solution to enable Stable Diffusion inference on an embedded DSP platform that lacked GPU or dedicated NPU acceleration. While Stable Diffusion comprises approximately 0.5 billion parameters across the Text Encoder, U-Net, and VAE Decoder, deploying such a model on DSP hardware presented significant challenges, including limited on-chip memory, constrained memory bandwidth, high computational complexity, compiler compatibility issues, and efficient coordination of model execution across multiple processing cores.
Solution Highlights
Vedya Labs implemented a DSP-optimized Stable Diffusion deployment framework on the client’s Axera AX650N platform featuring dual Cadence Tensilica Vision Q7 DSP cores. The implementation focused on optimizing the complete inference pipeline while maximizing hardware utilization within the constraints of the embedded architecture.
DSP-Optimized Stable Diffusion Pipeline
The complete Stable Diffusion workflow, including the Text Encoder, U-Net, and VAE Decoder, was optimized for efficient execution across the dual DSP architecture without relying on GPU or NPU acceleration.
Dual DSP Workload Partitioning
The U-Net, which accounts for the majority of the computational workload, was partitioned across two Vision Q7 DSP cores to enable parallel execution and significantly reduce inference latency.
Model Optimization & Quantization
The original FP32 models were converted to INT8 precision and optimized using layer fusion and XNNC compiler optimizations to reduce computational complexity while maintaining image generation quality.
Memory-Aware Execution
Chunked attention, block-wise computation scheduling, optimized memory movement, and efficient shared DDR utilization were implemented to minimize scratch memory usage and enable deployment within the available hardware resources.
Compiler & Runtime Integration
The deployment pipeline incorporated ONNX graph preprocessing, model surgery for XNNC compatibility, optimized DSP kernels, runtime integration, and RPC-based orchestration to enable seamless execution on the target platform.
DSP-Centric System Architecture
The implementation mapped the primary AI workloads to the dual DSP cores, while the Cortex-A55 CPU was utilized for orchestration, memory management, and runtime coordination, ensuring efficient utilization of the heterogeneous architecture.
Performance Achieved
The optimized implementation enabled Stable Diffusion inference with approximately 30 seconds prompt-to-image generation for 10 diffusion iterations, while reducing U-Net execution time to approximately 3 seconds, demonstrating the feasibility of deploying Generative AI workloads on DSP-centric embedded hardware.
Conclusion
By combining model optimization, compiler co-design, memory-aware execution, and DSP-specific runtime integration, Vedya Labs successfully enabled Stable Diffusion inference on the client’s dual Cadence Tensilica DSP platform without GPU or NPU acceleration. The implementation demonstrated that complex Generative AI workloads can be efficiently executed on DSP-centric embedded architectures, providing the client with a scalable foundation for future edge AI deployments.