===================================================================== CERT-Renater Note d'Information No. 2023/VULN030 _____________________________________________________________________ DATE : 27/01/2023 HARDWARE PLATFORM(S): / OPERATING SYSTEM(S): Systems running rancher versions prior to 2.5.17, 2.6.10, 2.7.1. ===================================================================== https://github.com/rancher/rancher/security/advisories/GHSA-34p5-jp77-fcrc https://github.com/rancher/rancher/security/advisories/GHSA-cq4p-vp5q-4522 https://github.com/rancher/rancher/security/advisories/GHSA-g25r-gvq3-wrq7 https://github.com/rancher/rancher/security/advisories/GHSA-8c69-r38j-rpfj https://github.com/rancher/rancher/security/advisories/GHSA-c45c-39f6-6gw9 https://github.com/rancher/rancher/security/advisories/GHSA-7m72-mh5r-6j3r _____________________________________________________________________ Command injection in Rancher Git package High severity GitHub Reviewed Published Jan 25, 2023 in rancher/rancher Vulnerability details Package github.com/rancher/rancher (Go) Affected versions > = 2.5.0, < 2.5.17 > = 2.6.0, < 2.6.10 > = 2.7.0, < 2.7.1 Patched versions 2.5.17 2.6.10 2.7.1 Description Impact An issue was discovered in Rancher from versions 2.5.0 up to and including 2.5.16, 2.6.0 up to and including 2.6.9 and 2.7.0, where a command injection vulnerability is present in the Rancher Git package. This package uses the underlying Git binary available in the Rancher container image to execute Git operations. Specially crafted commands, when not properly disambiguated, can cause confusion when executed through Git, resulting in command injection in the underlying Rancher host. This issue can potentially be exploited in Rancher in two ways: Adding an untrusted Helm catalog, in the Catalogs menu, that contains maliciously designed repo URL configuration in Helm charts. Modifying the URL configuration used to download KDM (Kontainer Driver Metadata) releases. By default, only the Rancher admin has permission to manage both configurations for the local cluster (the cluster where Rancher is provisioned). Note: More information about this category of issue in version control system (VCS) tools are available in Snyk's blog post. Workarounds Except for only adding trusted catalogs and the KDM URL to Rancher, there is no other workaround besides updating Rancher to a patched version. Patches Patched versions include releases 2.5.17, 2.6.10, 2.7.1 and later versions. It is also important to update to a patched version in case Rancher or its standalone Git package implementation is used as a Go library instead of the application itself. Otherwise, this vulnerability might affect your dependent code. For more information If you have any questions or comments about this advisory: Reach out to SUSE Rancher Security team for security related inquiries. Open an issue in Rancher repository. Verify our support matrix and product support lifecycle. References GHSA-34p5-jp77-fcrc Last updated Jan 25, 2023 Reviewed Jan 25, 2023 Published to the GitHub Advisory Database Jan 25, 2023 @macedogm macedogm published to rancher/rancher Jan 25, 2023 Severity High 7.6/ 10 CVSS base metrics Attack vector Network Attack complexity High Privileges required High User interaction Required Scope Changed Confidentiality High Integrity High Availability High CVSS:3.1/AV:N/AC:H/PR:H/UI:R/S:C/C:H/I:H/A:H Weaknesses CWE-77 CWE-78 CWE-88 CVE ID CVE-2022-43758 GHSA ID GHSA-34p5-jp77-fcrc Source code rancher/rancher Credits @snoopysecurity snoopysecurity _____________________________________________________________________ Plaintext storage of sensitive data in Rancher API and cluster.management.cattle.io objects Critical macedogm published GHSA-cq4p-vp5q-4522 Package rancher/rancher (Go) Affected versions > = 2.5.0, < 2.5.17 > = 2.6.0, < 2.6.10 > = 2.7.0, < 2.7.1 Patched versions 2.5.17 2.6.10 2.7.1 Description Impact This issue affects Rancher versions from 2.5.0 up to and including 2.5.16, from 2.6.0 up to and including 2.6.9 and 2.7.0. It was discovered that the security advisory CVE-2021-36782 (GHSA-g7j7-h4q8-8w2f), previously released by Rancher, missed addressing some sensitive fields, secret tokens, encryption keys, and SSH keys that were still being stored in plaintext directly on Kubernetes objects like Clusters. The exposed credentials are visible in Rancher to authenticated Cluster Owners, Cluster Members, Project Owners and Project Members of that cluster on the endpoints: /v1/management.cattle.io.cluster /v1/management.cattle.io.clustertemplaterevisions The remaining sensitive fields are now stripped from Clusters and other objects and moved to a Secret before the object is stored. The Secret is retrieved when the credential is needed. For objects that existed before this security fix, a one-time migration happens on startup. The fields that have been addressed by this security fix are: Cluster.Spec.RancherKubernetesEngineConfig.Services.KubeAPI.SecretsEncryptionConfig.CustomConfig.Providers[].AESGCM.Keys[].Secret Cluster.Spec.RancherKubernetesEngineConfig.Services.KubeAPI.SecretsEncryptionConfig.CustomConfig.Providers[].AESCBC.Keys[].Secret Cluster.Spec.RancherKubernetesEngineConfig.Services.KubeAPI.SecretsEncryptionConfig.CustomConfig.Providers[].SecretboxConfiguration.Keys[].Secret Cluster.Spec.RancherKubernetesEngineConfig.Services.Kubelet.ExtraEnv when containing the AWS_SECRET_ACCESS_KEY environment variable Cluster.Spec.RancherKubernetesEngineConfig.BastionHost.SSHKey Cluster.Spec.RancherKubernetesEngineConfig.PrivateRegistries[].ECRCredentialPlugin.AwsSecretAccessKey Cluster.Spec.RancherKubernetesEngineConfig.PrivateRegistries[].ECRCredentialPlugin.AwsSessionToken Cluster.Spec.RancherKubernetesEngineConfig.Network.AciNetworkProvider.ApicUserKey Cluster.Spec.RancherKubernetesEngineConfig.Network.AciNetworkProvider.KafkaClientKey Cluster.Spec.RancherKubernetesEngineConfig.Network.AciNetworkProvider.Token Important: For the exposure of credentials not related to Rancher, the final impact severity for confidentiality, integrity and availability is dependent on the permissions the leaked credentials have on their services. It is recommended to review for potentially leaked credentials in this scenario and to change them if deemed necessary. Workarounds There is no direct mitigation besides updating Rancher to a patched version. Patches Patched versions include releases 2.5.17, 2.6.10, 2.7.1 and later versions. After upgrading to a patched version, it is important to check for the ACISecretsMigrated and RKESecretsMigrated conditions on Clusters and ClusterTemplateRevisions to confirm when secrets have been fully migrated off of those objects, and the objects scoped within them. For more information If you have any questions or comments about this advisory: Reach out to SUSE Rancher Security team for security related inquiries. Open an issue in Rancher repository. Verify our support matrix and product support lifecycle. Severity Critical 9.9/ 10 CVSS base metrics Attack vector Network Attack complexity Low Privileges required Low User interaction None Scope Changed Confidentiality High Integrity High Availability High CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H CVE ID CVE-2022-43757 Weaknesses CWE-200 CWE-256 _____________________________________________________________________ Authenticated user can gain unauthorized shell pod and kubectl access in the local cluster High macedogm published GHSA-g25r-gvq3-wrq7 Package rancher/rancher (Go) Affected versions > = 2.5.0, < 2.5.17 > = 2.6.0, < 2.6.10 > = 2.7.0, < 2.7.1 Patched versions 2.5.17 2.6.10 2.7.1 Description Impact An issue was discovered in Rancher where an authorization logic flaw allows an authenticated user on any downstream cluster to (1) open a shell pod in the Rancher local cluster and (2) have limited kubectl access to it. The expected behavior is that a user does not have such access in the Rancher local cluster unless explicitly granted. This issue does not allow the user to escalate privileges in the local cluster directly (this would require another vulnerability to be exploited). The security issue happens in two different ways: Shell pod access - This is when a user opens a shell pod in the Rancher UI to a downstream cluster that the user has permission to access. The web request can be intercepted using the browser's web inspector/network console or a proxy tool to change the shell's destination to the Rancher local cluster instead of the desired downstream cluster. This flaw cannot be exploited to access a downstream cluster that the user has no permissions to. The shell pod runs with a limited non-root user, reducing the severity of this issue. However, even as a non-root user, it is still possible download and run binaries inside the shell pod. The blast radius of this issue can increase based on the configuration of the local cluster. For example: If the local cluster has unlimited network access, e.g. to the Internet, the user can open a reverse network connection to the shell pod. Or access the cloud metadata API of the underlying cloud infrastructure, where the user can extract the credentials associated with the local cluster and use them to interact with the cloud environment (this will be limited by the permissions granted to the cloud credentials in question). Check further recommendations about liming access to the cloud metadata API in Rancher's security best practices. Kubectl access - When downloading the kubeconfig file of a downstream cluster that the user has access to, the server cluster address in the kubeconfig file can be changed to point to the Rancher local cluster instead of the intended downstream cluster. This can also be achieved by crafting a kubeconfig using a Rancher token instead of using the kubeconfig from an active cluster. This flaw cannot be exploited to access a downstream cluster that the user has no permissions to. Notes: Rancher local cluster means the cluster where Rancher is installed. It is named as local inside the list of clusters in the Rancher UI. Audit logs in Rancher can be used to identify possible abuses of this issue, by tracking API requests to the user ID of the user that performed the action. API audit logs can be enabled as described in the documentation when set to level 1 or above. Workarounds There is no workaround or direct mitigation besides updating to a patched Rancher version. Patches Patched versions include releases 2.5.17, 2.6.10, 2.7.1 and later versions. For more information If you have any questions or comments about this advisory: Reach out to SUSE Rancher Security team for security related inquiries. Open an issue in Rancher repository. Verify our support matrix and product support lifecycle. Severity High 7.4/ 10 CVSS base metrics Attack vector Network Attack complexity Low Privileges required Low User interaction None Scope Changed Confidentiality Low Integrity Low Availability Low CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:L/I:L/A:L CVE ID CVE-2022-21953 Weaknesses CWE-284 CWE-285 _____________________________________________________________________ Rancher cattle-token is predictable High macedogm published GHSA-8c69-r38j-rpfj Package rancher/rancher (Go) Affected versions > = 2.6.0, < 2.6.10 > = 2.7.0, < 2.7.1 Patched versions 2.6.10 2.7.1 Description Impact An issue was discovered in Rancher versions up to and including 2.6.9 and 2.7.0, where the cattle-token secret, used by the cattle-cluster-agent, is predictable. Even after the token is regenerated, it will have the same value. This issue is not present in Rancher 2.5 releases. The cattle-token is used by Rancher's cattle-cluster-agent to connect to the Kubernetes API of Rancher provisioned downstream clusters. The problem occurs because the cattle-token secret does not use any random value in its composition, which causes it to always be regenerated with the same value. This can pose a serious problem if the token is compromised and needs to be recreated for security purposes. The usage of the cattle-token by an unauthorized user allows to escalate privileges to the cluster owner of the affected downstream cluster. It does not allow access to Rancher's own local cluster (the cluster where Rancher is provisioned). Workarounds In case it is not possible to promptly update to a patched version, a workaround is to use the rotate script provided in the public security advisory CVE-2021-36782 / GHSA-g7j7-h4q8-8w2f, which facilitates the rotation and creation of a new unique downstream cluster token. Patches Patched versions include releases 2.6.10, 2.7.1 and later versions. After upgrading to one of the patched versions, it is highly recommended to rotate the cattle-token in downstream clusters to guarantee that a new random token will be safely regenerated. The procedure below can rotate the cattle-token and should be executed in each downstream cluster provisioned by Rancher. It is recommended to first test this process in an appropriate development/testing environment. # Verify the current secret before rotating it $ kubectl describe secrets cattle-token -n cattle-system # Delete the secret $ kubectl delete secrets cattle-token -n cattle-system # Restart the cattle-cluster-agent deployment $ kubectl rollout restart deployment/cattle-cluster-agent -n cattle-system # Confirm that a new and different secret was generated $ kubectl describe secrets cattle-token -n cattle-system For more information If you have any questions or comments about this advisory: Reach out to SUSE Rancher Security team for security related inquiries. Open an issue in Rancher repository. Verify our support matrix and product support lifecycle. Severity High 7.1/ 10 CVSS base metrics Attack vector Network Attack complexity High Privileges required Low User interaction None Scope Unchanged Confidentiality High Integrity Low Availability High CVSS:3.1/AV:N/AC:H/PR:L/UI:N/S:U/C:H/I:L/A:H CVE ID CVE-2022-43755 Weaknesses CWE-330 _____________________________________________________________________ Rancher generated tokens not revoked after modifications made to authentication provider High macedogm published GHSA-c45c-39f6-6gw9 Package rancher/rancher (Go) Affected versions > = 2.5.0, < 2.5.17 > = 2.6.0, < 2.6.10 > = 2.7.0, < 2.7.1 Patched versions 2.5.17 2.6.10 2.7.1 Description Impact This issue affects Rancher versions from 2.5.0 up to and including 2.5.16, from 2.6.0 up to and including 2.6.9 and 2.7.0. It only affects Rancher setups that have an external authentication provider configured or had one configured in the past. It was discovered that when an external authentication provider is configured in Rancher and then disabled, the Rancher generated tokens associated with users who had access granted through the now disabled auth provider are not revoked. This allows users to retain *access to Rancher and kubectl access to clusters managed by Rancher, according to their previous configured permissions, even after they are supposed to have lost it due to the auth provider been disabled. The problem also occurs if the auth provider is configured (and is still enabled) to use the access level scopes allow members of clusters and projects, plus authorized users & groups and restrict access to only the authorized users & groups. In this case, removing users and groups from the authorized lists will not revoke the access tokens and they will remain valid. An example scenario is: OpenLDAP, MS Active Directory (AD) or any other external authentication provider is configured as an auth provider. A user (cluster-owner) is granted cluster-owner permissions on a downstream cluster (test-cluster). cluster-owner logs in using their external auth provider username and password. cluster-owner generates a kubeconfig token for test-cluster. The configured external auth provider is disabled. In this scenario, the kubeconfig generated in step 4 will still be valid after step 5, and test-cluster can still be accessed using the kubeconfig token. By default, tokens for authenticated session have their ttl (time to live) set to 960 minutes, so they will expire after 16 hours. kubeconfig tokens are configured to never expire, and their ttl is set to 0. These configurations can be changed in the Rancher's settings (Configuration > Global Settings > Settings) with the parameters auth-user-session-ttl-minutes and kubeconfig-default-token-ttl-minutes, respectively. Workarounds If you cannot update to a patched Rancher version, the recommended workaround is to review and remove tokens associated with auth providers manually. The tokens can be reviewed by executing kubectl get tokens in Rancher's local cluster. Each found token must be manually reviewed to check if it belongs to a user from a disabled auth provider or a user who's access was previously removed from the auth provider (when the auth provider is still enabled and is or was configured to use access level scopes, as mentioned above). The identified tokens can be removed with kubectl delete tokens . It is important to mention that this workaround must be done every time an auth provider is disabled in case you cannot update to a patched version. Patches Patched versions include releases 2.5.17, 2.6.10, 2.7.1 and later versions. After updating to a patched version, it is highly recommended to review the existing tokens and remove tokens related to disabled auth providers as described above in the workaround section. For more information If you have any questions or comments about this advisory: Reach out to SUSE Rancher Security team for security related inquiries. Open an issue in Rancher repository. Verify our support matrix and product support lifecycle. Severity High 8.8/ 10 CVSS base metrics Attack vector Network Attack complexity Low Privileges required Low User interaction None Scope Unchanged Confidentiality High Integrity High Availability High CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H CVE ID No known CVE Weaknesses CWE-287 _____________________________________________________________________ Privilege escalation in project role template binding (PRTB) and -promoted roles High macedogm published GHSA-7m72-mh5r-6j3r Package rancher/rancher (Go) Affected versions > = 2.5.0, < 2.5.17 > = 2.6.0, < 2.6.10 Patched versions 2.5.17 2.6.10 Description Impact An issue was discovered in Rancher versions from 2.5.0 up to and including 2.5.16 and from 2.6.0 up to and including 2.6.9, where an authorization logic flaw allows privilege escalation via project role template binding (PRTB) and -promoted roles. This issue is not present in Rancher 2.7 releases. Note: Consult Rancher documentation for more information about cluster and project roles and KB 000020097 for information about -promoted roles. This privilege escalation is possible for users with access to the escalate verb on PRTBs (projectroletemplatebindings.management.cattle.io), including users with * verbs on PRTBs (see notes below for more information). These users can escalate permissions for any -promoted resource (see the table below for a full enumeration) in any cluster where they have a PRTB granting such permissions in at least one project in the cluster. On a default Rancher setup, only the following roles have such permissions: Project Owner Manage Project Members These roles have permissions to affect the following resources: Resource API Group Affected Rancher version navlinks ui.cattle.io 2.6 nodes "" 2.6 persistentvolumes "" 2.5, 2.6 persistentvolumes core 2.5, 2.6 storageclasses storage.k8s.io 2.5, 2.6 apiservices apiregistration.k8s.io 2.5, 2.6 clusterrepos catalog.cattle.io 2.5, 2.6 clusters (local only) management.cattle.io 2.5, 2.6 Notes: During the calculation of the CVSS score, privileges required was considered as high because, by default, standard user and user-base users in Rancher do not have create, patch and update permissions on roletemplates. If a role template with access to those objects was already created by another user in the cluster, then this issue can be exploited by users without the mentioned permissions from point 1. Workarounds If updating Rancher to a patched version is not possible, then the following workarounds must be observed to mitigate this issue: Only grant Project Owner and Manage Project Members roles to trusted users. Minimize the creation of custom roles that contain the escalate, * or write verbs (create, delete, patch, update) on projectroletemplatebindings resource, and only grant such custom roles to trusted users. Minimize the number of users that have permissions to create, patch and update roletemplates. Patches Patched versions include releases 2.5.17 and 2.6.10 and later versions. This issue is not present in Rancher 2.7 releases. Detection The following script was developed to list role template bindings that give written access to the affected resources listed above. It is highly recommended to run the script in your environment and review the list of identified roles and role template bindings for possible signs of exploitation of this issue. The script requires jq installed and a kubeconfig with access to Rancher local cluster; it can also be executed in Rancher's kubectl shell. #!/bin/bash help=" Usage: bash find_promoted_resource.sh \n \n Requires: \n - jq installed and on path \n - A kubeconfig pointing at rancher's local cluster (can also run from rancher's kubectl shell) \n \n Outputs a list of roletemplates and roletemplate bindings which give write access to promoted resources. " if [[ $1 == "-h" || $1 == "--help" ]] then echo -e $help exit 0 fi # first, get the current roletemplates so that we only issue a get once kubectl get roletemplates.management.cattle.io -o json >> script_templates.json # find roles which have write access to a promoted resource. Filter on roleTemplates which fulfill all requirements: # Have a project context # Have some rules # Have one/more of the target api groups, or a * in the api groups # Have one/more of the target resources, or a * in the resources # Have a verb that is not read access (i.e. a verb that is not get/list/watch) roles=$(jq --argjson apiGroups '["", "ui.cattle.io", "core", "storage.k8s.io", "apiregistration.k8s.io", "catalog.cattle.io", "management.cattle.io"]' --argjson resources '["navlinks", "persistentvolumes", "nodes", "storageclasses", "apiservices", "clusterrepos", "clusters"]' --argjson verbs '["get", "list", "watch"]' '.items[] | select(.context=="project" and (.rules | length >= 1)) | select( .rules[] | select( (($apiGroups - .apiGroups | length < 7) or (.apiGroups | index("*"))) and (($resources - .resources | length < 7) or (.resources | index("*"))) and (.verbs - $verbs | length > 0)) | length >= 1 ) | .metadata.name' script_templates.json | jq -s ) # log promoted roles which give direct write access so they can be easily fixed echo "The following role templates give direct write access to a promoted resource:" echo $roles echo -e "" # find any roles which inherit first-level roles. Mostly a BFS which radiates outward from the known bad roles old_roles="[]" new_roles="$roles" old_length=$(echo $old_roles | jq 'length') new_length=$(echo $new_roles | jq 'length') # if our last loop found nothing new, it's safe to stop while [[ $old_length != $new_length ]]; do # set old values to what we currently know about old_roles=$new_roles old_length=$new_length # update new values with anything that inherits a "bad" role we know about new_roles=$(jq --argjson roles "$old_roles" --argjson roleLen "$old_length" '.items[] | .metadata.name as $NAME | select (( $roles | index($NAME)) or ((.roleTemplateNames | length > 0 ) and ($roles - .roleTemplateNames | length < $roleLen))) | .metadata.name ' script_templates.json | jq -s) new_length=$(echo $new_roles | jq 'length') done roles=$new_roles # log all roles which can give write access, even if it's not first level echo -e "The following role templates give write access to a promoted resource directly or through inheritance:" echo $roles echo -e "" kubectl get projectroletemplatebindings.management.cattle.io -A -o json >> script_bindings.json role_template_bindings=$(jq --argjson roleTemplates "$roles" '.items[] | .roleTemplateName as $TemplateName | select($roleTemplates | index($TemplateName)) | .metadata.name' script_bindings.json | jq -s) # since these bindings could be for users or groups, we need to include all fields which could help identify the subject. But they won't all be present, which makes the list look less pretty echo -e "The following is a list of bindings which give access to promoted resource, with the format of: bindingName, projectName, userName, userPrincipalName, groupName, groupPrincipalName: " echo $(jq --argjson bindings "$role_template_bindings" '.items[] | .metadata.name as $BindingName | select ( $bindings | index($BindingName)) | .metadata.name, .projectName, .userName?, .userPrincipalName?, .groupName?, .groupPrincipalName?' script_bindings.json | jq -s) unset old_roles unset new_roles unset roles unset role_template_bindings rm script_templates.json rm script_bindings.json For more information If you have any questions or comments about this advisory: Reach out to SUSE Rancher Security team for security related inquiries. Open an issue in Rancher repository. Verify our support matrix and product support lifecycle. Severity High 7.2/ 10 CVSS base metrics Attack vector Network Attack complexity Low Privileges required High User interaction None Scope Unchanged Confidentiality High Integrity High Availability High CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:H/I:H/A:H CVE ID CVE-2022-43759 Weaknesses CWE-269 ========================================================= + CERT-RENATER | tel : 01-53-94-20-44 + + 23/25 Rue Daviel | fax : 01-53-94-20-41 + + 75013 Paris | email:cert@support.renater.fr + =========================================================