RK3288 android 6.0 user release 相关问题处理

之前一直用的 userdebug 版本,默认 adb root 或者使用 su 即可获取 root 权限。目前基于安全考虑,固件发布需要使用 user 版本配置,很多功能都需要重新测试,下面对 RK3288 user 发布版本遇到的问题进行总结。

编译固件

首先是编译固件,将原先的 rk3288-userdebug 改为 rk3288-user.

./FFTools/make.sh -d rk3288-tb_8846 -j16 -l rk3288-user
./FFTools/mkupdate/mkupdate.sh -l rk3288-user

下面是 firefly 官方 wiki 文档对 user, userdebug, eng 版本的对比。

构建变体说明

默认的目标构建变体(TARGET_BUILD_VARIANT)为 userdebug。常用变体有三种,分别是用户(user)、用户调试(userdebug)和工程模式(eng),其区别如下:

  • user
    • 仅安装标签为 user 的模块
    • 设定属性 ro.secure=1,打开安全检查功能
    • 设定属性 ro.debuggable=0,关闭应用调试功能
    • 默认关闭 adb 功能
    • 打开 Proguard 混淆器
    • 打开 DEXPREOPT 预先编译优化
  • userdebug
    • 安装标签为 user、debug 的模块
    • 设定属性 ro.secure=1,打开安全检查功能
    • 设定属性 ro.debuggable=1,启用应用调试功能
    • 默认打开 adb 功能
    • 打开 Proguard 混淆器
    • 打开 DEXPREOPT 预先编译优化
  • eng
    • 安装标签为 user、debug、eng 的模块
    • 设定属性 ro.secure=0,关闭安全检查功能
    • 设定属性 ro.debuggable=1,启用应用调试功能
    • 设定属性 ro.kernel.android.checkjni=1,启用 JNI 调用检查
    • 默认打开 adb 功能
    • 关闭 Proguard 混淆器
    • 关闭 DEXPREOPT 预先编译优化

如果目标构建变体为 user,则 adb 无法获取 root 权限。

从以上说明可知,user 版本无法通过 adb 获取 root 权限,这个先不管,下面来看看我遇到了哪些问题。

adb shell fail

首先遇到的是 adb shell 失败,windows 和 ubuntu系统都测试失败。

  • ubuntu 执行 adb shell
$ adb shell
error: device unauthorized.
This adb server's $ADB_VENDOR_KEYS is not set
Try 'adb kill-server' if that seems wrong.
Otherwise check for a confirmation dialog on your device.
  • windows 执行 adb shell
C:\Users\EDY>adb shell
error: device unauthorized. Please check the confirmation dialog on your device.

提示信息类似,都是未认证,对比正常情况,手机插入电脑,然后打开调试功能是会弹出秘钥认证的,而 rk3288 系统没有弹框提示秘钥相关信息。最后排查发现是 ro.adb.secure 默认配置为0了,这个是 userdebug 版本中添加的,代表不需要通过认证,可以直接访问系统,但是这个在 user 版本是行不通的,所以需要把它改成1.

solution

修改 build/core/main.mk, 启用debug和adb认证功能。

diff --git a/core/main.mk b/core/main.mk
index aa617c3..a0b783d 100644
--- a/core/main.mk
+++ b/core/main.mk
@@ -361,9 +361,11 @@ ifeq (true,$(strip $(enable_target_debugging)))
   INCLUDE_TEST_OTA_KEYS := true
 else # !enable_target_debugging
   # Target is less debuggable and adbd is off by default
-  ADDITIONAL_DEFAULT_PROPERTIES += ro.debuggable=0
+  ADDITIONAL_DEFAULT_PROPERTIES += ro.debuggable=1
 endif # !enable_target_debugging

+ADDITIONAL_DEFAULT_PROPERTIES += ro.adb.secure=1
+
 ## eng ##

 ifeq ($(TARGET_BUILD_VARIANT),eng)

对于 ro.adb.secure , 也可以通过修改 device/rockchip/rk3288/system.prop 实现。

ro.adb.secure=1

fw version

release 版本的版本号显示如下,对应 ro.build.display.id ,为了和之前的版本号保持一致,需要更新。

MXC89K release-keys

solution

更新方式也简单,之前 userdebug 使用的是 $(build_desc), 所以在 user 版本的编译配置参数中也改为 $(build_desc).

--- a/core/Makefile
+++ b/core/Makefile
@@ -165,7 +165,7 @@ ifeq ($(TARGET_BUILD_VARIANT),user)
   ifeq "true" "$(DISPLAY_BUILD_NUMBER)"
     BUILD_DISPLAY_ID := $(BUILD_ID).$(BUILD_NUMBER) $(BUILD_KEYS)
   else
-    BUILD_DISPLAY_ID := $(BUILD_ID) $(BUILD_KEYS)
+    BUILD_DISPLAY_ID := $(build_desc)
   endif
 else
   # Non-user builds should show detailed build information

change usb mode fail

使用 user release 版本后,USB 模式无法修改了,一直保持在 Device 模式。修改触发的错误信息如下:

[   54.422452] type=1400 audit(1640315085.300:22): avc: denied { write } for pid=601 comm="ndroid.systemui" name="force_usb_mode" dev="sysfs" ino=12816 scontext=u:r:platform_app:s0:c512,c768 tcontext=u:object_r:sysfs:s0 tclass=file permissive=0
[   54.466011] init: avc:  denied  { set } for property=persist.usb.mode scontext=u:r:system_app:s0 tcontext=u:object_r:default_prop:s0 tclass=property_service
[   54.466052] init: sys_prop: permission denied uid:1000  name:persist.usb.mode


[   57.748360] type=1400 audit(1640315088.680:23): avc: denied { write } for pid=601 comm="ndroid.systemui" name="force_usb_mode" dev="sysfs" ino=12816 scontext=u:r:platform_app:s0:c512,c768 tcontext=u:object_r:sysfs:s0 tclass=file permissive=0
[   57.750643] init: avc:  denied  { set } for property=persist.usb.mode scontext=u:r:system_app:s0 tcontext=u:object_r:default_prop:s0 tclass=property_service
[   57.750696] init: sys_prop: permission denied uid:1000  name:persist.usb.mode
  • 与USB模式修改相关的文件
system/displayd/OtgManager.cpp
device/rockchip/common/init.rk30board.rc
device/rockchip/common/sepolicy/property_contexts
external/sepolicy/platform_app.te
external/sepolicy/app.te
  • 与USB模式修改相关的配置
ro.adb.secure=1
ro.debuggable
persist.usb.mode

参考博客 Android SELinux avc dennied权限问题解决方法 , 分析错误log

[   54.422452] type=1400 audit(1640315085.300:22): avc: denied { write } for pid=601 comm="ndroid.systemui" name="force_usb_mode" dev="sysfs" ino=12816 scontext=u:r:platform_app:s0:c512,c768 tcontext=u:object_r:sysfs:s0 tclass=file permissive=0
  • 缺失权限:denied { write }
  • 缺失权限的进程:scontext=u:r:platform_app
  • 访问的文件:tcontext=u:object_r:sysfs:s0
  • 访问的文件类型:tclass=file

所以需要在 external/sepolicy/platform_app.te中或者 device/rockchip/common/sepolicy/platform_app.te 中添加

allow platform_app sysfs:file write;

或者

allow platform_app sysfs:file rwx_file_perms;

但是添加后编译失败,因为这个权限是默认不允许 (neverallow) 的,log如下:

libsepol.report_failure: neverallow on line 353 of external/sepolicy/app.te (or line 7601 of policy.conf) violated by allow platform_app sysfs:file { write };
libsepol.check_assertions: 1 neverallow failures occurred
Error while expanding policy
external/sepolicy/Android.mk:69: recipe for target 'out/target/product/rk3288/obj/ETC/sepolicy_intermediates/sepolicy' failed
make: *** [out/target/product/rk3288/obj/ETC/sepolicy_intermediates/sepolicy] Error 1
make: *** Waiting for unfinished jobs....

根据log提示,打开文件 external/sepolicy/app.te 353 行,将 neverallow 的部分注释掉,或者取消 platform_app 的限制,再次编译就可以了。

solution

根据以上分析结果,添加权限。

diff --git a/app.te b/app.te
index 40de074..68ec6dd 100755
--- a/app.te
+++ b/app.te
@@ -349,7 +349,7 @@ neverallow appdomain efs_file:dir_file_class_set write;
 neverallow { appdomain -shell } efs_file:dir_file_class_set read;

 # Write to various pseudo file systems.
-neverallow { appdomain -bluetooth -nfc }
+neverallow { appdomain -bluetooth -nfc -platform_app }
     sysfs:dir_file_class_set write;
 neverallow appdomain
     proc:dir_file_class_set write;
diff --git a/platform_app.te b/platform_app.te
index 2afe4d8..c523de3 100644
--- a/platform_app.te
+++ b/platform_app.te
@@ -41,3 +41,4 @@ allow platform_app radio_service:service_manager find;
 allow platform_app surfaceflinger_service:service_manager find;
 allow platform_app app_api_service:service_manager find;
 allow platform_app system_api_service:service_manager find;
+allow platform_app sysfs:file write;

set persist.usb.mode fail

以上解决了 platform_app 的权限问题,下面还有一个权限问题。

[   54.466011] init: avc:  denied  { set } for property=persist.usb.mode scontext=u:r:system_app:s0 tcontext=u:object_r:default_prop:s0 tclass=property_service
[   54.466052] init: sys_prop: permission denied uid:1000  name:persist.usb.mode

solution

属性 persist.usb.mode 无法被 set, 这个不一定要修改 system_app.te, 可以在 sepolicy/property_contexts 中添加以下信息:

diff --git a/sepolicy/property_contexts b/sepolicy/property_contexts
index 8593ab2..7e113f1 100755
--- a/sepolicy/property_contexts
+++ b/sepolicy/property_contexts
@@ -3,3 +3,4 @@ sys_graphic.            u:object_r:graphic_prop:s0
 #ro.serialno             u:object_r:serialno_prop:s0
 drm.                    u:object_r:drm_prop:s0
 media.                  u:object_r:media_prop:s0
+persist.usb.            u:object_r:system_prop:s0

user_service.sh Permission denied

user_service.sh 是我们添加的一个开机启动脚本,开机log中提示 user_service.sh 不能执行,没有权限。

[   90.229367] init: cannot execve('/system/bin/user_service.sh'): Permission denied
[   90.229966] type=1400 audit(1640583793.920:23): avc: denied { entrypoint } for pid=1166 comm="init" path="/system/bin/user_service.sh" dev="mmcblk0p10" ino=462 scontext=u:r:shell:s0 tcontext=u:object_r:system_file:s0 tclass=file permissive=0

solution

修改 external/sepolicy/{domain.te,shell.te}, 启用 shellentrypoint 权限。

diff --git a/domain.te b/domain.te
index 2b30bd7..25b5ffb 100755
--- a/domain.te
+++ b/domain.te
@@ -251,7 +251,7 @@ neverallow { domain -init } kernel:security setsecparam;
 neverallow { domain -init -system_server -ueventd } hw_random_device:chr_file *;
 # Ensure that all entrypoint executables are in exec_type.
-neverallow domain { file_type -exec_type }:file entrypoint;
+neverallow { domain -shell } { file_type -exec_type }:file entrypoint;

 # Ensure that nothing in userspace can access /dev/mem or /dev/kmem
 neverallow { domain -kernel -ueventd -init } kmem_device:chr_file *;
diff --git a/shell.te b/shell.te
index 4b4093d..0350b7a 100644
--- a/shell.te
+++ b/shell.te
@@ -37,7 +37,7 @@ allow shell tty_device:chr_file rw_file_perms;
 allow shell console_device:chr_file rw_file_perms;
 allow shell input_device:dir r_dir_perms;
 allow shell input_device:chr_file rw_file_perms;
-allow shell system_file:file x_file_perms;
+allow shell system_file:file { entrypoint x_file_perms };
 allow shell shell_exec:file rx_file_perms;
 allow shell zygote_exec:file rx_file_perms;

ping fail after 4G/wifi network enabled

user_service.sh 在后台监听网络状态,状态修改后会自动添加路由,更新iptables等,但是 user 版本很多指令执行失败了,导致与本地连接的算法板之间的网络不通。而且切换网络(如关闭、打开wifi)时会打印以下权限问题。

$ [  423.709562] type=1400 audit(1640656107.794:490): avc: denied { write } for pid=3901 comm="pppd" name="ppp" dev="mmcblk0p13" ino=342742 scontext=u:r:rild:s0 tcontext=u:object_r:system_data_file:s0 tclass=dir permissive=0
[  426.563145] type=1400 audit(1640656110.694:491): avc: denied { write } for pid=3901 comm="pppd" name="ppp" dev="mmcblk0p13" ino=342742 scontext=u:r:rild:s0 tcontext=u:object_r:system_data_file:s0 tclass=dir permissive=0
[  426.620393] init: avc:  denied  { set } for property=net.interfaces.defaultroute scontext=u:r:rild:s0 tcontext=u:object_r:system_prop:s0 tclass=property_service
[  426.620467] init: sys_prop: permission denied uid:0  name:net.interfaces.defaultroute
[  426.641853] init: avc:  granted  { set } for property=net.ppp0.dns1 scontext=u:r:rild:s0 tcontext=u:object_r:net_radio_prop:s0 tclass=property_service
[  426.660313] init: avc:  granted  { set } for property=net.ppp0.dns2 scontext=u:r:rild:s0 tcontext=u:object_r:net_radio_prop:s0 tclass=property_service
[  426.676997] init: avc:  granted  { set } for property=net.ppp0.local-ip scontext=u:r:rild:s0 tcontext=u:object_r:net_radio_prop:s0 tclass=property_service
[  426.693011] init: avc:  granted  { set } for property=net.ppp0.remote-ip scontext=u:r:rild:s0 tcontext=u:object_r:net_radio_prop:s0 tclass=property_service
[  426.704633] init: avc:  granted  { set } for property=net.ppp0.gw scontext=u:r:rild:s0 tcontext=u:object_r:net_radio_prop:s0 tclass=property_service
[  427.055027] init: avc:  denied  { set } for property=persist.telephony.test.singleDc scontext=u:r:rild:s0 tcontext=u:object_r:default_prop:s0 tclass=property_service
[  427.055347] init: sys_prop: permission denied uid:0  name:persist.telephony.test.singleDc
[  606.939187] ==> rtl8188e_iol_efuse_patch
[  607.175605] RTL871X: nolinked power save leave
[  608.502840] RTL871X: nolinked power save enter

solution

这个问题非常麻烦,因为是 user 版本,没有root权限调试,绝大部分指令都无法使用,根本无法调试,所以我花费了一定时间将su集成进 user版本,详情见另一篇专门说明如何在 user 版本添加 su文档

集成 su 后,发现能够进入 root 状态了,但是 android 6.0 root用户也不是无所不能的,由于 sepolicy 的导入,导致很多权限都没有,于是只能一个个的查看log,然后修改 .te 文件,修改的文件主要包含以下几个。

device/rockchip/common/sepolicy/shell.te
device/rockchip/common/sepolicy/rild.te
device/rockchip/common/sepolicy/property_contexts
external/sepolicy/shell.te
external/sepolicy/domain.te
external/sepolicy/app.te

补丁如下

  • external/sepolicy
diff --git a/app.te b/app.te
index 5f8f648..419ecce 100755
--- a/app.te
+++ b/app.te
@@ -213,7 +213,7 @@ selinux_check_context(appdomain)

 # Superuser capabilities.
 # bluetooth requires net_admin and wake_alarm.
-neverallow { appdomain -bluetooth } self:capability *;
+neverallow { appdomain -bluetooth -shell } self:capability *;
 neverallow { appdomain -bluetooth } self:capability2 *;

 # Block device access.
@@ -349,7 +349,7 @@ neverallow appdomain efs_file:dir_file_class_set write;
 neverallow { appdomain -shell } efs_file:dir_file_class_set read;

 # Write to various pseudo file systems.
-neverallow { appdomain -bluetooth -nfc -platform_app }
+neverallow { appdomain -shell -bluetooth -nfc -platform_app }
     sysfs:dir_file_class_set write;
 neverallow appdomain
     proc:dir_file_class_set write;
diff --git a/domain.te b/domain.te
index 25b5ffb..ad765e7 100755
--- a/domain.te
+++ b/domain.te
@@ -338,7 +338,7 @@ neverallow domain default_android_service:service_manager add;

 # Require that domains explicitly label unknown properties, and do not allow
 # anyone but init to modify unknown properties.
-neverallow { domain -init } default_prop:property_service set;
+neverallow { domain -init -rild } default_prop:property_service set;

 neverallow { domain -init -recovery -system_server } frp_block_device:blk_file rw_file_perms;

@@ -398,7 +398,7 @@ neverallow domain { file_type fs_type dev_type }:{ lnk_file fifo_file sock_file
 # Nobody should be able to execute su on user builds.
 # On userdebug/eng builds, only dumpstate, shell, and
 # su itself execute su.
-neverallow { domain userdebug_or_eng(`-dumpstate -shell -su') } su_exec:file no_x_file_perms;
+# neverallow { domain userdebug_or_eng(`-dumpstate -shell -su') } su_exec:file no_x_file_perms;

 # Do not allow the introduction of new execmod rules. Text relocations
 # and modification of executable pages are unsafe.
diff --git a/shell.te b/shell.te
index 0350b7a..836f9fd 100644
--- a/shell.te
+++ b/shell.te
@@ -40,6 +40,8 @@ allow shell input_device:chr_file rw_file_perms;
 allow shell system_file:file { entrypoint x_file_perms };
 allow shell shell_exec:file rx_file_perms;
 allow shell zygote_exec:file rx_file_perms;
+allow shell su_exec:file rx_file_perms;
+allow shell shell:capability { net_admin net_raw dac_override setgid setuid };
  • device/rockchip/common
diff --git a/sepolicy/property_contexts b/sepolicy/property_contexts
index 7e113f1..530d8aa 100755
--- a/sepolicy/property_contexts
+++ b/sepolicy/property_contexts
@@ -4,3 +4,4 @@ sys_graphic.            u:object_r:graphic_prop:s0
 drm.                    u:object_r:drm_prop:s0
 media.                  u:object_r:media_prop:s0
 persist.usb.            u:object_r:system_prop:s0
+net.interfaces.         u:object_r:system_prop:s0
diff --git a/sepolicy/rild.te b/sepolicy/rild.te
index a798f14..9cfb015 100644
--- a/sepolicy/rild.te
+++ b/sepolicy/rild.te
@@ -16,6 +16,8 @@ unix_socket_connect(rild, rpc_reg, rpc);
 # ppp for 3G dongle 
 allow rild ppp_device:chr_file rw_file_perms;
 allow rild net_radio_prop:property_service set;
+allow rild default_prop:property_service set;
+allow rild system_prop:property_service set;
 allow rild ppp_exec:file {execute execute_no_trans open};
 allow rild ppp_exec:file getattr;
 allow rild ppp_exec:file { read };
@@ -25,3 +27,4 @@ allow rild toolbox_exec:file {getattr execute_no_trans execute read open};
 #allow rild system_data_file:dir {write add_name};
 #allow rild system_data_file:file {create write};
 allow rild self:capability { sys_admin };
+allow rild system_data_file:dir create_dir_perms;
diff --git a/sepolicy/shell.te b/sepolicy/shell.te
index 7d1999c..8c2aeb1 100755
--- a/sepolicy/shell.te
+++ b/sepolicy/shell.te
@@ -1,3 +1,4 @@
+permissive shell;
 # Additional rules for shell
 allow shell toolbox_exec:file { read getattr open execute execute_no_trans };
 allow shell gpsd_exec:file { getattr execute read open execute_no_trans };
@@ -7,3 +8,9 @@ allow shell serial_device:chr_file rw_file_perms;
 allow shell proc_cpuinfo:file mounton;
 allow shell surfaceflinger:fifo_file rw_file_perms;
 allow shell bootanim_exec:file getattr;
+allow shell proc_net:file rw_file_perms;
+allow shell self:udp_socket ioctl;
+allow shell self:netlink_route_socket nlmsg_write;
+allow shell net_data_file:dir search;
+allow shell net_data_file:file rw_file_perms;
+allow shell sysfs:file rw_file_perms

权限问题解决后,发现启用 wifi、4G 网络后还是无法ping通算法板,于是开始慢慢调试路。

使用 iptables ip rule 等指令排查网络问题,发现wifi,4G网络启用后,ping 算法板会默认使用 wifi4G 网络对应的链路,这说明路由出问题了。

然后我用 ip rule 去查策略路由,对比 userdebug 版本,发现少了一个 table 1, 这个table是用来添加算法板的默认路由的,而且优先级比较高,现在没有了。接着我找到了添加 table 1 的脚本,是 user_service.sh 中的函数 add_route_table

add_route_table() {                                                                                                           
    PREF=$(ip rule list | busybox head -n 2 | busybox tail -n 1 | busybox awk '{print $1}' | busybox sed 's/://g')
    let PREF--                                                                                                           
    ip rule add from all table 1 pref ${PREF}
}

我在shell中执行了以上指令,发现 busybox 找不到!好吧, user 版本没有编译 busybox, 那就把它去掉,逐个测试 head, tail, awk, sed 指令,发现 awk 指令也无法执行。

为了解决这个问题,我先根据指令去解析其意图,发现这条指令是为了获取 ip rule list 中第二行的前缀,也就是优先级对应的数值 10000

0:      from all lookup local
10000:  from all fwmark 0xc0000/0xd0000 lookup legacy_system
13000:  from all fwmark 0x10063/0x1ffff lookup local_network
15000:  from all fwmark 0x0/0x10000 lookup legacy_system
16000:  from all fwmark 0x0/0x10000 lookup legacy_network
17000:  from all fwmark 0x0/0x10000 lookup local_network
23000:  from all fwmark 0x0/0xffff uidrange 0-0 lookup main
32000:  from all unreachable

既如此,解决方案就简单了,根本不需要 awk, sed, 直接用 cut 就好了,patch如下。

 add_route_table() {                                                                                                           
-    PREF=$(ip rule list | busybox head -n 2 | busybox tail -n 1 | busybox awk '{print $1}' | busybox sed 's/://g')
+    PREF=$(ip rule list | head -n 2 | tail -n 1 | cut -d ':' -f1)
     let PREF--                                                                                                           
     ip rule add from all table 1 pref ${PREF}
 }

reference