Tensor Compression and Reconstruction in Split DNN for Real-time Object Detection at the Edge

Paper in proceeding, 2024

Computer vision applications for UAVs often rely on deep neural networks (DNNs) to increase prediction accuracy. As such DNNs crave for computational resources that can be hardly matched by those available at the UAVs, an emerging solution is to split the DNN into a low-complexity head model run at the UAV and a heavier tail run at the edge. This approach, however, comes at the cost of transmitting a large tensor data, hence of high bandwidth consumption, on the UAV-edge radio link. We tackle this problem by proposing the Compressed Tensor-based DNN split (CoTeD) framework, which executes tensor compression at the UAV and reconstruction at the edge, while conveniently trading off tensor compression with quality of the computer vision task output. When compared with the no-split case, CoTeD reduces the UAV computational burden by 50% w.r.t. performing inference at the UAV only, and the amount of transmitted data by over one order of magnitude w.r.t. running inference at the edge only. When compared to compressive sensing, JPEG-100, and the whole DNN run at the edge, CoTeD decreases the overall latency by (resp.) 95%, 75%, and 80%.