Abstract: There has been a highly hopeful advancement in image interpretation and sequencing through computer vision and therefore video camera has come to be a very essential sensor for applications such as economic traffic monitoring and surveillance. But extraction of features from high dimensional database is lesser during detection stage. This makes detection and tracking of multiple vehicles existing in same video based traffic surveillance a huge issue. This study is intended to overcome such issues of vehicles detection and tracking of multiple vehicles that are present in front of camera. The major contributions of proposed approach include background subtraction, vehicle detection and vehicle tracking. In background subtraction methods, models are employed to implement over background intensities in order to overcome minor changes in environment. Vehicle tracking stages have been conducted in two stages in which first stage is concerned with extraction of significant features like symmetry, edge, headlight, brightness and appearance during day and night time as got from Improved Particle Swarm Optimization (IPSO) algorithm then followed by dimensionality reduction of features by means of Hybrid Principal Component Analysis (HPCA). The second stage is involved with Fuzzy Hybrid Information Inference Mechanism (FHIIM) for determining tracked vehicles as reported in previous work. In vehicle detection stage, candidate vehicles resulting from background subtraction have been detected. The proposed approach has been evaluated by performing experiments with case studies of vehicles. Experimental results have shown that the proposed system performs better even during the situations of congestion.