Signal is centralized, closed-source, not-selfhostable and requires being attached to a phone number.
Matrix is decentralized, federated, fully open source with multiple client and server implementations, self-hostable, and does not require being attached to a phone number.
Where the metadata goes I think is important as well.
All Signal metadata necessarily goes through Signal’s servers and is tied to your phone number, but not all Matrix metadata ever gets near the Matrix.org if you are using a different homeserver.
I think both are less than ideal in that regard, and I think Briar (strictly P2P) has a much better model for dealing with this at the expense of generally being a UX disaster.