Think sudoku without the cages. Each row and column contains one of each number 1-6.
Appreciate that this walkthrough covers one path of logic, even though there may be multiple legitimate logic paths.
The 1’s on the board are not adjacent to another 1; the other four 1’s are. The 1 in the fifth column cannot go in the top row, obviously. If it’s above the 5, it won’t be adjacent to another 1. It can’t go diagonally to the other 1 either, since that 1 is not adjacent to another 1. So the 1 in Column 5 goes in the highlighted cell. This of course means that another 1 is down-right from that 1.
As for the 1 in Row 2, it has to go in Column 3 so as not to be adjacent to the corner 1. Then the final 1 must be Row 3, Column 2.
The main diagonal is obviously not the one that has a 1 in it. Since the highlighted cell is prime but not 2 or 5, it must be 3.
You can now finish that row. The highlighted cell cannot be 4, so it is 6. The 4 then is in Column 3.
The remaining corners must be 2, 3, and 4. Two of those are on the prime-only diagonal, so the highlighted cell must be 4. The far right column already has a 2, so the top right corner is 3 and the bottom left corner is 2.
The highlighted cell must be prime, but there’s a 5 in the same column. Now we know that 2’s cannot be adjacent to each other, so highlighted cell cannot be 2 either. So it is 3.
Now you can get a bunch. The highlighted cell can no longer be 3, so it is 5. The other cell in the left column is 3 then.
Row 6, Column 5 can only be 6. Right above it cannot be 4 (same row), so it is 2. So the top cell in that column is 4.
In the far right column, the second from top cannot be 5 (same row), so it is 6. Third from bottom, then, is 5.
The highlighted cell is on the prime-only diagonal. It cannot be 3 or 5 (same row), so it is 2.
The highlighted cell or the one to its right must be a 3. The cell next to the 1 cannot be adjacent to a 4, since that row and the one above it already have a 4. So the highlighted cell must be 3, meaning the one next to it is 5.
The highlighted cell must be prime but not 3 (same column) or 2 (same row), so it is 5. What’s left for that row is 6 and 3, so 3 is in the third cell and 6 in the fourth.
The highlighted cell cannot be 1, 2, 3, 5, 6. It is 4, meaning the other number in that row is 6.
The highlighted cell cannot be 1, 3, 4, 5, 6, so it is 2. The other cell in that row, then, is 4.
The last three cells are obvious, since their columns are all 5/6 filled.