We all should be using (and improving) the schedutil governor now. Get
rid of the non-upstream governor.
Tested on Hikey.
Change-Id: I2104558b03118b0a9c5f099c23c42cd9a6c2a963
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Before commit 5f8b3a757d65 ("sched/fair: consider task utilization in
group_norm_util()"), eenv->util_delta is used to distinguish energy
before and energy after in sched_group_energy(). After that commit,
eenv->util_delta can not do that any more.
In this commit, use trg_cpu to distinguish energy before/after in
sched_group_energy().
Before apply this commit, cap_before/cap_delta is not correct:
<idle>-0 [001] 147504.608920: sched_energy_diff: pid=7 comm=rcu_preempt
src_cpu=1 dst_cpu=3 usage_delta=7 nrg_before=250 nrg_after=250 nrg_diff=0
cap_before=0 cap_after=528 cap_delta=1056 nrg_delta=0 nrg_payoff=0
After apply this commit, cap_before/cap_delta retrun to normal:
<idle>-0 [001] 220.494011: sched_energy_diff: pid=7 comm=rcu_preempt
src_cpu=1 dst_cpu=2 usage_delta=3 nrg_before=248 nrg_after=248 nrg_diff=0
cap_before=528 cap_after=528 cap_delta=0 nrg_delta=0 nrg_payoff=0
Change-Id: I7b5f7ccce56e93af7ea4e87d8e0ea6e2405f9c27
Signed-off-by: Ke Wang <ke.wang@spreadtrum.com>
(cherry picked from commit 0da783a605cd20d5a37c2a840e8a1fa641c09768)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
If no energy saving for target_cpu in the calculation of energy_diff(),
backup_cpu will be set as the new dst_cpu for the next calculation. At this
point, we also need update the new trg_cpu as backup_cpu to make sure the
subsequent calculation of energy_diff() is correct.
Change-Id: If3b35b6dc54865f1cb4b1603134102d4422227d5
Signed-off-by: Ke Wang <ke.wang@spreadtrum.com>
(cherry picked from commit b1923e22f4eca3e537e015d6ea3dce1187edea37)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
The group_max_util() function is used to compute the maximum utilization
across the CPUs of a certain energy_env configuration.
Its main client is the energy_diff function when it needs to compute the
SG capacity for one of the before/after scheduling candidates.
Currently, the energy_diff function sets util_delta = 0 when it wants to
compute the energy corresponding to the scheduling candidate where the
task runs in the previous CPU. This implies that, for the task waking up
in the previous CPU we consider only its blocked load tracked by the CPU
RQ. However, in case of a medium-big task which is waking up on a long
time idle CPU, this blocked load can be already completely decayed.
More in general, the current approach is biased towards under-estimating
the capacity requirements for the "before" scheduling candidate.
This patch fixes this by:
- always use the cpu_util_wake() to properly get the utilization of a CPU
without any (partially decayed) contribution of the waking up task
- adding the task utilization to the cpu_util_wake just for the target
cpu
The "target CPU" is defined by the energy_env to be either the src_cpu or
the dst_cpu, depending on which scheduling candidate we are considering.
Finally, since this update removes the last usage of calc_util_delta()
this function is now safely removed.
Change-Id: I20ee1bcf40cee6bf6e265fb2d32ef79061ad6ced
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
(cherry picked from commit 52d70152fade678e304683bd1a5842af95e83558)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
The group_norm_util() function is used to compute the normalized
utilization of a SG given a certain energy_env configuration.
The main client of this function is the energy_diff function when it
comes to compute the SG energy for one of the before/after scheduling
candidates.
Currently, the energy_diff function sets util_delta = 0 when it wants to
compute the energy corresponding to the scheduling candidate where the
task runs in the previous CPU. This implies that, for the task waking up
in the previous CPU we consider only its blocked load tracked by the CPU
RQ. However, in case of a medium-big task which is waking up on a long
time idle CPU, this blocked load can be already completely decayed.
More in general, the current approach is biased towards under-estimating
the energy consumption for the "before" scheduling candidate.
This patch fixes this by:
- always use the cpu_util_wake() to properly get the utilization of a CPU
without any (partially decayed) contribution of the waking up task
- adding the task utilization to the cpu_util_wake just for the
target cpu
The "target CPU" is defined by the energy_env to be either the src_cpu
or the dst_cpu, depending on which scheduling candidate we are
considering.
This patch update also the definition of __cpu_norm_util(), which is
currently called just by the group_norm_util() function. This allows to
simplify the code by using this function just to normalize a specified
utilization with respect to a given capacity.
This update allows to completely remove any dependency of
group_norm_util() from calc_util_delta().
Change-Id: I3b6ec50ce8decb1521faae660e326ab3319d3c82
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
(cherry picked from commit ef34ea830347ca175ba8a1baf05357ed98d5728c)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
For non latency sensitive tasks the goal is to optimize for energy efficiency.
Thus, we should try our best to avoid moving a task on a CPU which is then
going to be marked as overutilized.
Let's use the capacity_margin metric to verify if a candidate target CPU
should be considered without risking to bail out of EAS mode.
Change-Id: Ib3697106f4073aedf4a6c6ce42bd5d000fa8c007
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
(cherry picked from commit f95753da4ba7e1c1eee0500ffe41b3e5fa68b347)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
The find_best_target can sometimes not return a valid backup CPU, either
because it cannot find one or just becasue it returns prev_cpu as a backup.
In these cases we should skip the energy_diff evaluation for the backup CPU.
Change-Id: I3787dbdfe74122348dd7a7485b88c4679051bd32
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
(cherry picked from commit d9bcf5b88594a0225b51878236e49305f272eadc)
[trivial cherry-pick issue]
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
In systems where SchedTune is enabled, we do not report energy diff for non
boosted tasks. Let's fix this by always genereting an energy_diff event where
however:
nrg.delta = 0, since we skip energy normalization
payoff = nrg.diff, since the payoff is defined just by the energy difference
Change-Id: I9a11ec19b6f56da04147f5ae5b47daf1dd180445
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
(cherry picked from commit 13e2e3c7f7d09f619444631a0962fd8020660b8d)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
We use task_util() in find_idlest_group() via capacity_spare_wake().
This task_util() updated in wake_cap(). However wake_cap() is not the
only reason for ending up in find_idlest_group() - we could have been sent
there by wake_wide(). So explicitly sync the task util with prev_cpu
when we are about to head to find_idlest_group().
We could simply do this at the beginning of
select_task_rq_fair() (i.e. irrespective of whether we're heading to
select_idle_sibling() or find_idlest_group() & co), but I didn't want to
slow down the select_idle_sibling() path more than necessary.
Don't do this during fork balancing, we won't need the task_util and
we'd just clobber the last_update_time, which is supposed to be 0.
Change-Id: I56113d8d67cf338f3fb1a692422289cf659399a3
Signed-off-by: Brendan Jackman <brendan.jackman@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andres Oportus <andresoportus@google.com>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Joel Fernandes <joelaf@google.com>
Cc: Josef Bacik <josef@toxicpanda.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Link: http://lkml.kernel.org/r/20170808095519.10077-1-brendan.jackman@arm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
(cherry picked from commit ea16f0ea6c3d in tip:sched/core)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
Add the update_rq_clock() call at the top of the callstack instead of
at the bottom where we find it missing, this to aid later effort to
minimize the number of update_rq_lock() calls.
WARNING: CPU: 30 PID: 194 at ../kernel/sched/sched.h:797 assert_clock_updated()
rq->clock_update_flags < RQCF_ACT_SKIP
Call Trace:
dump_stack()
__warn()
warn_slowpath_fmt()
assert_clock_updated.isra.63.part.64()
can_migrate_task()
load_balance()
pick_next_task_fair()
__schedule()
schedule()
worker_thread()
kthread()
Change-Id: I17716141789f9fac709495951cd2e079cf49d6d8
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
(cherry picked from commit 3bed5e2166a5e433bf62162f3cd3c5174d335934)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
Instead of adding the update_rq_clock() all the way at the bottom of
the callstack, add one at the top, this to aid later effort to
minimize update_rq_lock() calls.
WARNING: CPU: 0 PID: 1 at ../kernel/sched/sched.h:797 detach_task_cfs_rq()
rq->clock_update_flags < RQCF_ACT_SKIP
Call Trace:
dump_stack()
__warn()
warn_slowpath_fmt()
detach_task_cfs_rq()
switched_from_fair()
__sched_setscheduler()
_sched_setscheduler()
sched_set_stop_task()
cpu_stop_create()
__smpboot_create_thread.part.2()
smpboot_register_percpu_thread_cpumask()
cpu_stop_init()
do_one_initcall()
? print_cpu_info()
kernel_init_freeable()
? rest_init()
kernel_init()
ret_from_fork()
Change-Id: Iee08c2ed3303ae8f0c527658f13646b02a412cad
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
(cherry picked from commit 80f5c1b84baa8180c3c27b7e227429712cd967b6)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
When using schedfreq on cpus with max capacity significantly smaller than
1024, the tick update uses non-normalised capacities - this leads to
selecting an incorrect OPP as we were scaling the frequency as if the
max capacity achievable was 1024 rather than the max for that particular
cpu or group. This could result in a cpu being stuck at the lowest OPP
and unable to generate enough utilisation to climb out if the max
capacity is significantly smaller than 1024.
Instead, normalize the capacity to be in the range 0-1024 in the tick
so that when we later select a frequency, we get the correct one.
Also comments updated to be clearer about what is needed.
Change-Id: Id84391c7ac015311002ada21813a353ee13bee60
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
(cherry picked from commit bc3b0db024f3d0b323e7d93994655e9e3d9f8d68)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
When we convert capacity into frequency, we used policy->max to get
the max freq of the cpu. Since this can be changed by userspace policy
or thermal events, we are potentially asking for a lower frequency
than the utilization demands.
Change over to using cpuinfo.max which is the max freq supported by
that cpu rather than the currently-chosen max. Frequency granted still
honours the max policy.
Tested by setting a userspace policy and observing the relevant vars
in a trace. In this instance, we ask for around 1ghz instead of 620MHz.
freq_new=1013512
unfixed_freq_new=624487
capacity=546
cpuinfo_max=1900800
policy_max=1171200
Change-Id: I8c5694db42243c6fb78bb9be9046b06ac81295e7
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
(cherry picked from commit 114c6ceb2e5b0e30442e2ba5f78cfaedf76bf1f9)
[trivial cherry-pick issue]
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
On systems using ACPI cpufreq, at boot-time only CPU0 max_freq has been
populated when load_scale_factor/capacity recomputation is triggered on
all possible cpus. This leads to the following divide by zero panic when
rq->max_freq for any cpu other than CPU0 is accessed:
[ 4.827597] divide error: 0000 [#1] PREEMPT SMP
[ 4.827604] [<ffffffff862ad88e>] ? notifier_call_chain+0x37/0x57
[ 4.827607] [<ffffffff862adaf3>] ? __blocking_notifier_call_chain+0x46/0x5c
[ 4.827611] [<ffffffff8684b9f4>] ? cpufreq_set_policy+0xa6/0x2dc
[ 4.827614] [<ffffffff869e4d55>] ? cpufreq_init_policy+0xba/0xde
[ 4.827617] [<ffffffff8684bd4c>] ? cpufreq_update_policy+0x122/0x122
[ 4.827620] [<ffffffff8684c51c>] ? cpufreq_online+0x5a7/0x64e
[ 4.827623] [<ffffffff8684c63b>] ? cpufreq_add_dev+0x78/0xed
[ 4.827627] [<ffffffff866ce412>] ? subsys_interface_register+0xd6/0x117
[ 4.827630] [<ffffffff8684b410>] ? cpufreq_register_driver+0x168/0x2b2
[ 4.827632] [<ffffffff8684b410>] ? cpufreq_register_driver+0x168/0x2b2
[ 4.827636] [<ffffffff86f83b2e>] ? cpufreq_gov_dbs_init+0xc/0xc
[ 4.827639] [<ffffffff86f83d43>] ? acpi_cpufreq_init+0x215/0x282
[ 4.827641] [<ffffffff86f83b2e>] ? cpufreq_gov_dbs_init+0xc/0xc
[ 4.827645] [<ffffffff8620046e>] ? do_one_initcall+0x9c/0x12e
[ 4.827649] [<ffffffff86f3d058>] ? kernel_init_freeable+0x190/0x22b
[ 4.827651] [<ffffffff869e59d9>] ? rest_init+0x7c/0x7c
[ 4.827653] [<ffffffff869e59e3>] ? kernel_init+0xa/0xf3
[ 4.827656] [<ffffffff869ed482>] ? ret_from_fork+0x22/0x30
On ACPI based systems, for CPUs with uninitialized rq->max_freq skip the
re-calculation.
Reference discussion on a similar issue here:
https://www.spinics.net/lists/arm-kernel/msg557612.html
Change-Id: I5025e3f6db671e9e72369f85708d3b0482d5dad2
Signed-off-by: Abhilash Kesavan <abhilash.kesavan@intel.com>
The util returned from group_max_util is not capped at the max util
present in the group, so it can be larger than the capacity stored in
the array. Ensure that when this happens, we always use the last entry
in the array to fetch energy from.
Tested with synthetics on Juno board.
Change-Id: I89fb52fb7e68fa3e682e308acc232596672d03f7
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Make the schedutil governor take the initial (default) value of the
rate_limit_us sysfs attribute from the (new) transition_delay_us
policy parameter (to be set by the scaling driver).
That will allow scaling drivers to make schedutil use smaller default
values of rate_limit_us and reduce the default average time interval
between consecutive frequency changes.
Make intel_pstate set transition_delay_us to 500.
BACKPORT: Modified to support the separate up_rate_limit_us and
down_rate_limit_us (upstream just has a single rate_limit_us). Also
dropped the changes for intel_pstate as there's a merge conflict.
Change-Id: I62a8543879a4d8582cdcb31ebd55607705d1c8b1
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
(cherry picked from commit 1b72e7fd304639f1cd49d1e11955c4974936d88c)
Signed-off-by: Brendan Jackman <brendan.jackman@arm.com>
Changes in 4.9.58
MIPS: Fix minimum alignment requirement of IRQ stack
Revert "bsg-lib: don't free job in bsg_prepare_job"
xen-netback: Use GFP_ATOMIC to allocate hash
locking/lockdep: Add nest_lock integrity test
watchdog: kempld: fix gcc-4.3 build
irqchip/crossbar: Fix incorrect type of local variables
initramfs: finish fput() before accessing any binary from initramfs
mac80211_hwsim: check HWSIM_ATTR_RADIO_NAME length
ALSA: hda: Add Geminilake HDMI codec ID
qed: Don't use attention PTT for configuring BW
mac80211: fix power saving clients handling in iwlwifi
net/mlx4_en: fix overflow in mlx4_en_init_timestamp()
staging: vchiq_2835_arm: Make cache-line-size a required DT property
netfilter: nf_ct_expect: Change __nf_ct_expect_check() return value.
iio: adc: xilinx: Fix error handling
f2fs: do SSR for data when there is enough free space
sched/fair: Update rq clock before changing a task's CPU affinity
Btrfs: send, fix failure to rename top level inode due to name collision
f2fs: do not wait for writeback in write_begin
md/linear: shutup lockdep warnning
sparc64: Migrate hvcons irq to panicked cpu
net/mlx4_core: Fix VF overwrite of module param which disables DMFS on new probed PFs
crypto: xts - Add ECB dependency
mm/memory_hotplug: set magic number to page->freelist instead of page->lru.next
ocfs2/dlmglue: prepare tracking logic to avoid recursive cluster lock
slub: do not merge cache if slub_debug contains a never-merge flag
scsi: scsi_dh_emc: return success in clariion_std_inquiry()
ASoC: mediatek: add I2C dependency for CS42XX8
drm/amdgpu: refuse to reserve io mem for split VRAM buffers
net: mvpp2: release reference to txq_cpu[] entry after unmapping
qede: Prevent index problems in loopback test
qed: Reserve doorbell BAR space for present CPUs
qed: Read queue state before releasing buffer
i2c: at91: ensure state is restored after suspending
ceph: don't update_dentry_lease unless we actually got one
ceph: fix bogus endianness change in ceph_ioctl_set_layout
ceph: clean up unsafe d_parent accesses in build_dentry_path
uapi: fix linux/rds.h userspace compilation errors
uapi: fix linux/mroute6.h userspace compilation errors
IB/hfi1: Use static CTLE with Preset 6 for integrated HFIs
IB/hfi1: Allocate context data on memory node
target/iscsi: Fix unsolicited data seq_end_offset calculation
hrtimer: Catch invalid clockids again
nfsd/callback: Cleanup callback cred on shutdown
powerpc/perf: Add restrictions to PMC5 in power9 DD1
drm/nouveau/gr/gf100-: fix ccache error logging
regulator: core: Resolve supplies before disabling unused regulators
btmrvl: avoid double-disable_irq() race
EDAC, mce_amd: Print IPID and Syndrome on a separate line
cpufreq: CPPC: add ACPI_PROCESSOR dependency
usb: dwc3: gadget: Correct ISOC DATA PIDs for short packets
Linux 4.9.58
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
[ Upstream commit a499c3ead88ccf147fc50689e85a530ad923ce36 ]
This is triggered during boot when CONFIG_SCHED_DEBUG is enabled:
------------[ cut here ]------------
WARNING: CPU: 6 PID: 81 at kernel/sched/sched.h:812 set_next_entity+0x11d/0x380
rq->clock_update_flags < RQCF_ACT_SKIP
CPU: 6 PID: 81 Comm: torture_shuffle Not tainted 4.10.0+ #1
Hardware name: LENOVO ThinkCentre M8500t-N000/SHARKBAY, BIOS FBKTC1AUS 02/16/2016
Call Trace:
dump_stack+0x85/0xc2
__warn+0xcb/0xf0
warn_slowpath_fmt+0x5f/0x80
set_next_entity+0x11d/0x380
set_curr_task_fair+0x2b/0x60
do_set_cpus_allowed+0x139/0x180
__set_cpus_allowed_ptr+0x113/0x260
set_cpus_allowed_ptr+0x10/0x20
torture_shuffle+0xfd/0x180
kthread+0x10f/0x150
? torture_shutdown_init+0x60/0x60
? kthread_create_on_node+0x60/0x60
ret_from_fork+0x31/0x40
---[ end trace dd94d92344cea9c6 ]---
The task is running && !queued, so there is no rq clock update before calling
set_curr_task().
This patch fixes it by updating rq clock after holding rq->lock/pi_lock
just as what other dequeue + put_prev + enqueue + set_curr story does.
Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1487749975-5994-1-git-send-email-wanpeng.li@hotmail.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Sasha Levin <alexander.levin@verizon.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The "goto force_balance" here is intended to mitigate the fact that
avg_load calculations can result in bad placement decisions when
priority is asymmetrical.
The original commit that adds it:
fab476228b ("sched: Force balancing on newidle balance if local group has capacity")
explains:
Under certain situations, such as a niced down task (i.e. nice =
-15) in the presence of nr_cpus NICE0 tasks, the niced task lands
on a sched group and kicks away other tasks because of its large
weight. This leads to sub-optimal utilization of the
machine. Even though the sched group has capacity, it does not
pull tasks because sds.this_load >> sds.max_load, and f_b_g()
returns NULL.
A similar but inverted issue also affects ARM big.LITTLE (asymmetrical CPU
capacity) systems - consider 8 always-running, same-priority tasks on a
system with 4 "big" and 4 "little" CPUs. Suppose that 5 of them end up on
the "big" CPUs (which will be represented by one sched_group in the DIE
sched_domain) and 3 on the "little" (the other sched_group in DIE), leaving
one CPU unused. Because the "big" group has a higher group_capacity its
avg_load may not present an imbalance that would cause migrating a
task to the idle "little".
The force_balance case here solves the problem but currently only for
CPU_NEWLY_IDLE balances, which in theory might never happen on the
unused CPU. Including CPU_IDLE in the force_balance case means
there's an upper bound on the time before we can attempt to solve the
underutilization: after DIE's sd->balance_interval has passed the
next nohz balance kick will help us out.
Change-Id: I4cbc8ced11e6a56f98ec67633e67e09cbb515268
Signed-off-by: Brendan Jackman <brendan.jackman@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20170807163900.25180-1-brendan.jackman@arm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
(cherry picked from commit 583ffd99d7657755736d831bbc182612d1d2697d)
Signed-off-by: Brendan Jackman <brendan.jackman@arm.com>
find_idlest_group() only compares the runnable_load_avg when looking for
the least loaded group. But on fork intensive use case like hackbench
where tasks blocked quickly after the fork, this can lead to selecting the
same CPU instead of other CPUs, which have similar runnable load but a
lower load_avg.
When the runnable_load_avg of 2 CPUs are close, we now take into account
the amount of blocked load as a 2nd selection factor. There is now 3 zones
for the runnable_load of the rq:
-[0 .. (runnable_load - imbalance)] : Select the new rq which has
significantly less runnable_load
-](runnable_load - imbalance) .. (runnable_load + imbalance)[ : The
runnable loads are close so we use load_avg to chose between the 2 rq
-[(runnable_load + imbalance) .. ULONG_MAX] : Keep the current rq which
has significantly less runnable_load
The scale factor that is currently used for comparing runnable_load,
doesn't work well with small value. As an example, the use of a scaling
factor fails as soon as this_runnable_load == 0 because we always select
local rq even if min_runnable_load is only 1, which doesn't really make
sense because they are just the same. So instead of scaling factor, we use
an absolute margin for runnable_load to detect CPUs with similar
runnable_load and we keep using scaling factor for blocked load.
For use case like hackbench, this enable the scheduler to select different
CPUs during the fork sequence and to spread tasks across the system.
Tests have been done on a Hikey board (ARM based octo cores) for several
kernel. The result below gives min, max, avg and stdev values of 18 runs
with each configuration.
The v4.8+patches configuration also includes the changes below which is
part of the proposal made by Peter to ensure that the clock will be up to
date when the fork task will be attached to the rq.
@@ -2568,6 +2568,7 @@ void wake_up_new_task(struct task_struct *p)
__set_task_cpu(p, select_task_rq(p, task_cpu(p), SD_BALANCE_FORK, 0));
#endif
rq = __task_rq_lock(p, &rf);
+ update_rq_clock(rq);
post_init_entity_util_avg(&p->se);
activate_task(rq, p, 0);
hackbench -P -g 1
ea86cb4b767dc603c902 v4.8 v4.8+patches
min 0.049 0.050 0.051 0,048
avg 0.057 0.057(0%) 0.057(0%) 0,055(+5%)
max 0.066 0.068 0.070 0,063
stdev +/-9% +/-9% +/-8% +/-9%
Change-Id: I8fe40bbd106454282b1963db825705f3cf377b9e
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Tested-by: Matt Fleming <matt@codeblueprint.co.uk>
Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk>
During fork, the utilization of a task is init once the rq has been
selected because the current utilization level of the rq is used to set
the utilization of the fork task. As the task's utilization is still
null at this step of the fork sequence, it doesn't make sense to look for
some spare capacity that can fit the task's utilization.
Furthermore, I can see perf regressions for the test "hackbench -P -g 1"
because the least loaded policy is always bypassed and tasks are not
spread during fork.
With this patch and the fix below, we are back to same performances as
for v4.8. The fix below is only a temporary one used for the test until a
smarter solution is found because we can't simply remove the test which is
useful for others benchmarks
@@ -5708,13 +5708,6 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, int t
avg_cost = this_sd->avg_scan_cost;
- /*
- * Due to large variance we need a large fuzz factor; hackbench in
- * particularly is sensitive here.
- */
- if ((avg_idle / 512) < avg_cost)
- return -1;
-
time = local_clock();
for_each_cpu_wrap(cpu, sched_domain_span(sd), target, wrap) {
Change-Id: If7dd1c2a81fd74419733b8ecff519ae8cb8eb32d
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Acked-by: Morten Rasmussen <morten.rasmussen@arm.com>
Tested-by: Matt Fleming <matt@codeblueprint.co.uk>
Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk>
Changes in 4.9.55
USB: gadgetfs: Fix crash caused by inadequate synchronization
USB: gadgetfs: fix copy_to_user while holding spinlock
usb: gadget: udc: atmel: set vbus irqflags explicitly
usb: gadget: udc: renesas_usb3: fix for no-data control transfer
usb: gadget: udc: renesas_usb3: fix Pn_RAMMAP.Pn_MPKT value
usb: gadget: udc: renesas_usb3: Fix return value of usb3_write_pipe()
usb-storage: unusual_devs entry to fix write-access regression for Seagate external drives
usb-storage: fix bogus hardware error messages for ATA pass-thru devices
usb: renesas_usbhs: fix the BCLR setting condition for non-DCP pipe
usb: renesas_usbhs: fix usbhsf_fifo_clear() for RX direction
ALSA: usb-audio: Check out-of-bounds access by corrupted buffer descriptor
usb: pci-quirks.c: Corrected timeout values used in handshake
USB: cdc-wdm: ignore -EPIPE from GetEncapsulatedResponse
USB: dummy-hcd: fix connection failures (wrong speed)
USB: dummy-hcd: fix infinite-loop resubmission bug
USB: dummy-hcd: Fix erroneous synchronization change
USB: devio: Don't corrupt user memory
usb: gadget: mass_storage: set msg_registered after msg registered
USB: g_mass_storage: Fix deadlock when driver is unbound
USB: uas: fix bug in handling of alternate settings
USB: core: harden cdc_parse_cdc_header
usb: Increase quirk delay for USB devices
USB: fix out-of-bounds in usb_set_configuration
xhci: fix finding correct bus_state structure for USB 3.1 hosts
xhci: Fix sleeping with spin_lock_irq() held in ASmedia 1042A workaround
xhci: set missing SuperSpeedPlus Link Protocol bit in roothub descriptor
Revert "xhci: Limit USB2 port wake support for AMD Promontory hosts"
iio: adc: twl4030: Fix an error handling path in 'twl4030_madc_probe()'
iio: adc: twl4030: Disable the vusb3v1 rugulator in the error handling path of 'twl4030_madc_probe()'
iio: ad_sigma_delta: Implement a dedicated reset function
staging: iio: ad7192: Fix - use the dedicated reset function avoiding dma from stack.
iio: core: Return error for failed read_reg
IIO: BME280: Updates to Humidity readings need ctrl_reg write!
iio: ad7793: Fix the serial interface reset
iio: adc: mcp320x: Fix readout of negative voltages
iio: adc: mcp320x: Fix oops on module unload
uwb: properly check kthread_run return value
uwb: ensure that endpoint is interrupt
staging: vchiq_2835_arm: Fix NULL ptr dereference in free_pagelist
mm, oom_reaper: skip mm structs with mmu notifiers
lib/ratelimit.c: use deferred printk() version
lsm: fix smack_inode_removexattr and xattr_getsecurity memleak
ALSA: compress: Remove unused variable
Revert "ALSA: echoaudio: purge contradictions between dimension matrix members and total number of members"
ALSA: usx2y: Suppress kernel warning at page allocation failures
mlxsw: spectrum: Prevent mirred-related crash on removal
net: sched: fix use-after-free in tcf_action_destroy and tcf_del_walker
sctp: potential read out of bounds in sctp_ulpevent_type_enabled()
tcp: update skb->skb_mstamp more carefully
bpf/verifier: reject BPF_ALU64|BPF_END
tcp: fix data delivery rate
udpv6: Fix the checksum computation when HW checksum does not apply
ip6_gre: skb_push ipv6hdr before packing the header in ip6gre_header
net: phy: Fix mask value write on gmii2rgmii converter speed register
ip6_tunnel: do not allow loading ip6_tunnel if ipv6 is disabled in cmdline
net/sched: cls_matchall: fix crash when used with classful qdisc
tcp: fastopen: fix on syn-data transmit failure
net: emac: Fix napi poll list corruption
packet: hold bind lock when rebinding to fanout hook
bpf: one perf event close won't free bpf program attached by another perf event
isdn/i4l: fetch the ppp_write buffer in one shot
net_sched: always reset qdisc backlog in qdisc_reset()
net: qcom/emac: specify the correct size when mapping a DMA buffer
vti: fix use after free in vti_tunnel_xmit/vti6_tnl_xmit
l2tp: Avoid schedule while atomic in exit_net
l2tp: fix race condition in l2tp_tunnel_delete
tun: bail out from tun_get_user() if the skb is empty
net: dsa: Fix network device registration order
packet: in packet_do_bind, test fanout with bind_lock held
packet: only test po->has_vnet_hdr once in packet_snd
net: Set sk_prot_creator when cloning sockets to the right proto
netlink: do not proceed if dump's start() errs
ip6_gre: ip6gre_tap device should keep dst
ip6_tunnel: update mtu properly for ARPHRD_ETHER tunnel device in tx path
tipc: use only positive error codes in messages
net: rtnetlink: fix info leak in RTM_GETSTATS call
socket, bpf: fix possible use after free
powerpc/64s: Use emergency stack for kernel TM Bad Thing program checks
powerpc/tm: Fix illegal TM state in signal handler
percpu: make this_cpu_generic_read() atomic w.r.t. interrupts
driver core: platform: Don't read past the end of "driver_override" buffer
Drivers: hv: fcopy: restore correct transfer length
stm class: Fix a use-after-free
ftrace: Fix kmemleak in unregister_ftrace_graph
HID: i2c-hid: allocate hid buffers for real worst case
HID: wacom: leds: Don't try to control the EKR's read-only LEDs
HID: wacom: Always increment hdev refcount within wacom_get_hdev_data
HID: wacom: bits shifted too much for 9th and 10th buttons
rocker: fix rocker_tlv_put_* functions for KASAN
netlink: fix nla_put_{u8,u16,u32} for KASAN
iwlwifi: mvm: use IWL_HCMD_NOCOPY for MCAST_FILTER_CMD
iwlwifi: add workaround to disable wide channels in 5GHz
scsi: sd: Do not override max_sectors_kb sysfs setting
brcmfmac: add length check in brcmf_cfg80211_escan_handler()
brcmfmac: setup passive scan if requested by user-space
drm/i915/bios: ignore HDMI on port A
nvme-pci: Use PCI bus address for data/queues in CMB
mmc: core: add driver strength selection when selecting hs400es
sched/cpuset/pm: Fix cpuset vs. suspend-resume bugs
vfs: deny copy_file_range() for non regular files
ext4: fix data corruption for mmap writes
ext4: Don't clear SGID when inheriting ACLs
ext4: don't allow encrypted operations without keys
f2fs: don't allow encrypted operations without keys
KVM: x86: fix singlestepping over syscall
Linux 4.9.55
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
commit 50e76632339d4655859523a39249dd95ee5e93e7 upstream.
Cpusets vs. suspend-resume is _completely_ broken. And it got noticed
because it now resulted in non-cpuset usage breaking too.
On suspend cpuset_cpu_inactive() doesn't call into
cpuset_update_active_cpus() because it doesn't want to move tasks about,
there is no need, all tasks are frozen and won't run again until after
we've resumed everything.
But this means that when we finally do call into
cpuset_update_active_cpus() after resuming the last frozen cpu in
cpuset_cpu_active(), the top_cpuset will not have any difference with
the cpu_active_mask and this it will not in fact do _anything_.
So the cpuset configuration will not be restored. This was largely
hidden because we would unconditionally create identity domains and
mobile users would not in fact use cpusets much. And servers what do use
cpusets tend to not suspend-resume much.
An addition problem is that we'd not in fact wait for the cpuset work to
finish before resuming the tasks, allowing spurious migrations outside
of the specified domains.
Fix the rebuild by introducing cpuset_force_rebuild() and fix the
ordering with cpuset_wait_for_hotplug().
Reported-by: Andy Lutomirski <luto@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: <stable@vger.kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Tejun Heo <tj@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Fixes: deb7aa308e ("cpuset: reorganize CPU / memory hotplug handling")
Link: http://lkml.kernel.org/r/20170907091338.orwxrqkbfkki3c24@hirez.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit bc4278987e3874da62edf585fe8b3bdd9b53f638 upstream.
A regression of the FTQ noise has been reported by Ying Huang,
on the following hardware:
8 threads Intel(R) Core(TM)i7-4770 CPU @ 3.40GHz with 8G memory
... which was caused by this commit:
commit 4e5160766fcc ("sched/fair: Propagate asynchrous detach")
The only part of the patch that can increase the noise is the update
of blocked load of group entity in update_blocked_averages().
We can optimize this call and skip the update of group entity if its load
and utilization are already null and there is no pending propagation of load
in the task group.
This optimization partly restores the noise score. A more agressive
optimization has been tried but has shown worse score.
Reported-by: ying.huang@linux.intel.com
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: dietmar.eggemann@arm.com
Cc: ying.huang@intel.com
Fixes: 4e5160766fcc ("sched/fair: Propagate asynchrous detach")
Link: http://lkml.kernel.org/r/1489758442-2877-1-git-send-email-vincent.guittot@linaro.org
[ Fixed typos, improved layout. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Change-Id: Iba93bbda71125ce14519aa30b1beac2bf3c52e1e
Fixes: Change-Id: I5d1a9f273111c0bb7503ca3c9ad2406d12867cb5
("UPSTREAM: sched/fair: Propagate asynchrous detach")
Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
Changes in 4.9.41
af_key: Add lock to key dump
pstore: Make spinlock per zone instead of global
net: reduce skb_warn_bad_offload() noise
jfs: Don't clear SGID when inheriting ACLs
ALSA: fm801: Initialize chip after IRQ handler is registered
ALSA: hda - Add missing NVIDIA GPU codec IDs to patch table
parisc: Prevent TLB speculation on flushed pages on CPUs that only support equivalent aliases
parisc: Extend disabled preemption in copy_user_page
parisc: Suspend lockup detectors before system halt
powerpc/pseries: Fix of_node_put() underflow during reconfig remove
NFS: invalidate file size when taking a lock.
NFSv4.1: Fix a race where CB_NOTIFY_LOCK fails to wake a waiter
crypto: authencesn - Fix digest_null crash
KVM: PPC: Book3S HV: Enable TM before accessing TM registers
md/raid5: add thread_group worker async_tx_issue_pending_all
drm/vmwgfx: Fix gcc-7.1.1 warning
drm/nouveau/disp/nv50-: bump max chans to 21
drm/nouveau/bar/gf100: fix access to upper half of BAR2
KVM: PPC: Book3S HV: Restore critical SPRs to host values on guest exit
KVM: PPC: Book3S HV: Save/restore host values of debug registers
Revert "powerpc/numa: Fix percpu allocations to be NUMA aware"
Staging: comedi: comedi_fops: Avoid orphaned proc entry
drm: rcar-du: Simplify and fix probe error handling
smp/hotplug: Move unparking of percpu threads to the control CPU
smp/hotplug: Replace BUG_ON and react useful
nfc: Fix hangup of RC-S380* in port100_send_ack()
nfc: fdp: fix NULL pointer dereference
net: phy: Do not perform software reset for Generic PHY
isdn: Fix a sleep-in-atomic bug
isdn/i4l: fix buffer overflow
ath10k: fix null deref on wmi-tlv when trying spectral scan
wil6210: fix deadlock when using fw_no_recovery option
mailbox: always wait in mbox_send_message for blocking Tx mode
mailbox: skip complete wait event if timer expired
mailbox: handle empty message in tx_tick
sched/cgroup: Move sched_online_group() back into css_online() to fix crash
RDMA/uverbs: Fix the check for port number
ipmi/watchdog: fix watchdog timeout set on reboot
dentry name snapshots
v4l: s5c73m3: fix negation operator
pstore: Allow prz to control need for locking
pstore: Correctly initialize spinlock and flags
pstore: Use dynamic spinlock initializer
net: skb_needs_check() accepts CHECKSUM_NONE for tx
device-dax: fix sysfs duplicate warnings
x86/mce/AMD: Make the init code more robust
r8169: add support for RTL8168 series add-on card.
ARM: omap2+: fixing wrong strcat for Non-NULL terminated string
dt-bindings: power/supply: Update TPS65217 properties
dt-bindings: input: Specify the interrupt number of TPS65217 power button
ARM: dts: am57xx-idk: Put USB2 port in peripheral mode
ARM: dts: n900: Mark eMMC slot with no-sdio and no-sd flags
net/mlx5: Disable RoCE on the e-switch management port under switchdev mode
ipv6: Should use consistent conditional judgement for ip6 fragment between __ip6_append_data and ip6_finish_output
net/mlx4_core: Use-after-free causes a resource leak in flow-steering detach
net/mlx4: Remove BUG_ON from ICM allocation routine
net/mlx4_core: Fix raw qp flow steering rules under SRIOV
drm/msm: Ensure that the hardware write pointer is valid
drm/msm: Put back the vaddr in submit_reloc()
drm/msm: Verify that MSM_SUBMIT_BO_FLAGS are set
vfio-pci: use 32-bit comparisons for register address for gcc-4.5
irqchip/keystone: Fix "scheduling while atomic" on rt
ASoC: tlv320aic3x: Mark the RESET register as volatile
spi: dw: Make debugfs name unique between instances
ASoC: nau8825: fix invalid configuration in Pre-Scalar of FLL
irqchip/mxs: Enable SKIP_SET_WAKE and MASK_ON_SUSPEND
openrisc: Add _text symbol to fix ksym build error
dmaengine: ioatdma: Add Skylake PCI Dev ID
dmaengine: ioatdma: workaround SKX ioatdma version
l2tp: consider '::' as wildcard address in l2tp_ip6 socket lookup
dmaengine: ti-dma-crossbar: Add some 'of_node_put()' in error path.
usb: dwc3: omap: fix race of pm runtime with irq handler in probe
ARM64: zynqmp: Fix W=1 dtc 1.4 warnings
ARM64: zynqmp: Fix i2c node's compatible string
perf probe: Fix to get correct modname from elf header
ARM: s3c2410_defconfig: Fix invalid values for NF_CT_PROTO_*
ACPI / scan: Prefer devices without _HID/_CID for _ADR matching
usb: gadget: Fix copy/pasted error message
Btrfs: use down_read_nested to make lockdep silent
Btrfs: fix lockdep warning about log_mutex
benet: stricter vxlan offloading check in be_features_check
Btrfs: adjust outstanding_extents counter properly when dio write is split
Xen: ARM: Zero reserved fields of xatp before making hypervisor call
tools lib traceevent: Fix prev/next_prio for deadline tasks
xfrm: Don't use sk_family for socket policy lookups
perf tools: Install tools/lib/traceevent plugins with install-bin
perf symbols: Robustify reading of build-id from sysfs
video: fbdev: cobalt_lcdfb: Handle return NULL error from devm_ioremap
vfio-pci: Handle error from pci_iomap
arm64: mm: fix show_pte KERN_CONT fallout
nvmem: imx-ocotp: Fix wrong register size
net: usb: asix_devices: add .reset_resume for USB PM
ASoC: fsl_ssi: set fifo watermark to more reliable value
sh_eth: enable RX descriptor word 0 shift on SH7734
ARCv2: IRQ: Call entry/exit functions for chained handlers in MCIP
ALSA: usb-audio: test EP_FLAG_RUNNING at urb completion
x86/platform/intel-mid: Rename 'spidev' to 'mrfld_spidev'
perf/x86: Set pmu->module in Intel PMU modules
ASoC: Intel: bytcr-rt5640: fix settings in internal clock mode
HID: ignore Petzl USB headlamp
scsi: fnic: Avoid sending reset to firmware when another reset is in progress
scsi: snic: Return error code on memory allocation failure
scsi: bfa: Increase requested firmware version to 3.2.5.1
ASoC: Intel: Skylake: Release FW ctx in cleanup
ASoC: dpcm: Avoid putting stream state to STOP when FE stream is paused
Linux 4.9.41
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
commit 96b777452d8881480fd5be50112f791c17db4b6b upstream.
Commit:
2f5177f0fd ("sched/cgroup: Fix/cleanup cgroup teardown/init")
.. moved sched_online_group() from css_online() to css_alloc().
It exposes half-baked task group into global lists before initializing
generic cgroup stuff.
LTP testcase (third in cgroup_regression_test) written for testing
similar race in kernels 2.6.26-2.6.28 easily triggers this oops:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
IP: kernfs_path_from_node_locked+0x260/0x320
CPU: 1 PID: 30346 Comm: cat Not tainted 4.10.0-rc5-test #4
Call Trace:
? kernfs_path_from_node+0x4f/0x60
kernfs_path_from_node+0x3e/0x60
print_rt_rq+0x44/0x2b0
print_rt_stats+0x7a/0xd0
print_cpu+0x2fc/0xe80
? __might_sleep+0x4a/0x80
sched_debug_show+0x17/0x30
seq_read+0xf2/0x3b0
proc_reg_read+0x42/0x70
__vfs_read+0x28/0x130
? security_file_permission+0x9b/0xc0
? rw_verify_area+0x4e/0xb0
vfs_read+0xa5/0x170
SyS_read+0x46/0xa0
entry_SYSCALL_64_fastpath+0x1e/0xad
Here the task group is already linked into the global RCU-protected 'task_groups'
list, but the css->cgroup pointer is still NULL.
This patch reverts this chunk and moves online back to css_online().
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Fixes: 2f5177f0fd ("sched/cgroup: Fix/cleanup cgroup teardown/init")
Link: http://lkml.kernel.org/r/148655324740.424917.5302984537258726349.stgit@buzz
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Matt Fleming <matt@codeblueprint.co.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Sometimes we find a target cpu but then we do not use it as
the energy_diff indicates that we would increase energy usage
or not save anything. To offer an additional option for those
cases, we return a second option which is what we would have
selected if the target CPU had not been found. This gives us
another chance to try to save some energy.
Change-Id: I42c4f20aba10e4cf65b51ac4153e2e00e534c8c7
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
(cherry picked from commit 426e11af9e415437d857e0f3cc5215b43a7cee08)
[Fixed cherry-pick issue]
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
In the current EAS group energy calculations, we only use
the idle state of the group as it is right now. This means
that there are times when EAS cannot see that we are about
to remove all utilization from a group which is likely to
result in us being able to idle that entire group.
This is an attempt to detect that situation and at least
allow the energy calculation to include savings in that
scenario, regardless of what we might be able to actually
achieve in the real world. If a cluster or cpu looks like
it will have some idle time available to it, we try to
map the utilization onto an idle state.
Change-Id: I8fcb1e507f65ae6a2c5647eeef75a4bf28c7a0c0
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
(cherry picked from commit 6a43b3d6d4f0ce4c978a7b3170f130146a3dbc12)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
Before using a task's util_avg signal in EAS, we need to ensure that
it has been synced up to the last_update_time of prev_cpu's root
cfs_rq.
We previously relied on the side effect of wake_cap to do that,
however that does not happen when the waking CPU has the same
capacity as the prev_cpu. Therefore just explicitly call
sync_entity_load_avg. This may result in calling that function twice
within the same select_task_rq_fair, but since last_update_time
hasn't changed the second call will bail out very quickly.
Change-Id: I91f1fcd71dfeb96b7f5b73418f1cf9ac311d4655
Signed-off-by: Brendan Jackman <brendan.jackman@arm.com>
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
(cherry picked from commit 595d8e0d207b9dfb0aa412d24293ed80e6478c88)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
If there have misfit task on one CPU, current code does not handle this
situation for nohz idle balance. As result, we can see the misfit task
stays run on little core for long time.
So this patch check if the CPU has misfit task or not. If has misfit
task then kick nohz idle balance so finally can execute active balance.
Change-Id: I117d3b7404296f8de11cb960a87a6b9a54a9f348
Signed-off-by: Leo Yan <leo.yan at linaro.org>
[taken from https://lists.linaro.org/pipermail/eas-dev/2016-September/000551.html]
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
(cherry picked from commit 07bb98f399e2df79c79f6ad563fc69eff8c32967)
[Fixed cpu_load_update_idle call]
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
Stale cpu utilization signals can cause havoc for energy-aware systems,
and they are caused by no updates being performed for cpus which have
no tick running. There is open debate about when is the correct time to
update these cpus, and general recognition that something needs to be
done.
This is an attempt to do something useful.
When we are looking for a task to pull for a newly-idle cpu, we have
an opportunity to update the stats for any cpu which has no tick running
without causing too much disturbance to the system or waking it up.
Change-Id: I0280104ea9c53e56c26f1c56a62bacab5d3e951b
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Brendan Jackman <brendan.jackman@arm.com>
(cherry picked from commit 352ffffddc25e6237395210439f2c49e68606703)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
The find_best_target() code has evolved over time to integrate different
micro-optimizations to the point to be quite difficult now to follow
exactly what it's doing.
This patch rafactors the existing code to make it more readable and easy
to maintain. It does that by properly identifying the three main
use-cases and addressing them in priority order:
A) latency sensitive tasks
B) non latency sensitive tasks on IDLE CPUs
C) non latency sensitive tasks on ACTIVE CPUs
The original behaviors are preserved. Some tests to compare
power/performances before and after this patch have been done using
Jankbench and YouTube and we did not noticed sensible differences.
The only difference with respect of the original code is a small update
to favor lower-capacity idle CPUs in case B. The same preference is not
enforce in case A since this can lead to a selection of a non-reserved
CPU for TOP_APP tasks, which ultimately can lead to non desirable
co-scheduling side-effects.
Change-Id: I871e5d95af89176217e4e239b64d44a420baabe8
Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
(removed checkpatch whitespace error)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
[Fixed cherry-pick issue]
(cherry picked from commit 6b0008b7d7be7d141f9855337eb09ad0c9217cfb)
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
sugov_update_commit() calls trace_cpu_frequency() to record the
current CPU frequency if it has not changed in the fast switch case
to prevent utilities from getting confused (they may report that the
CPU is idle if the frequency has not been recorded for too long, for
example).
However, that may cause the tracepoint to be triggered quite often
for no real reason (if the frequency doesn't change, we will not
modify the last update time stamp and governor computations may
run again shortly when that happens), so don't do that (arguably, it
is done to work around a utilities bug anyway).
That allows code duplication in sugov_update_commit() to be reduced
somewhat too.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
(cherry picked from commit 38d4ea229d25d30be6bf41bcd6cd663a587866ca)
(conflicts with sugov_up_down_rate_limit resolved)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Change-Id: Ia019dda29b8c1c4cf3553da75c88d066eb5674e9
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
The way the schedutil governor uses the PELT metric causes it to
underestimate the CPU utilization in some cases.
That can be easily demonstrated by running kernel compilation on
a Sandy Bridge Intel processor, running turbostat in parallel with
it and looking at the values written to the MSR_IA32_PERF_CTL
register. Namely, the expected result would be that when all CPUs
were 100% busy, all of them would be requested to run in the maximum
P-state, but observation shows that this clearly isn't the case.
The CPUs run in the maximum P-state for a while and then are
requested to run slower and go back to the maximum P-state after
a while again. That causes the actual frequency of the processor to
visibly oscillate below the sustainable maximum in a jittery fashion
which clearly is not desirable.
That has been attributed to CPU utilization metric updates on task
migration that cause the total utilization value for the CPU to be
reduced by the utilization of the migrated task. If that happens,
the schedutil governor may see a CPU utilization reduction and will
attempt to reduce the CPU frequency accordingly right away. That
may be premature, though, for example if the system is generally
busy and there are other runnable tasks waiting to be run on that
CPU already.
This is unlikely to be an issue on systems where cpufreq policies are
shared between multiple CPUs, because in those cases the policy
utilization is computed as the maximum of the CPU utilization values
over the whole policy and if that turns out to be low, reducing the
frequency for the policy most likely is a good idea anyway. On
systems with one CPU per policy, however, it may affect performance
adversely and even lead to increased energy consumption in some cases.
On those systems it may be addressed by taking another utilization
metric into consideration, like whether or not the CPU whose
frequency is about to be reduced has been idle recently, because if
that's not the case, the CPU is likely to be busy in the near future
and its frequency should not be reduced.
To that end, use the counter of idle calls in the timekeeping code.
Namely, make the schedutil governor look at that counter for the
current CPU every time before its frequency is about to be reduced.
If the counter has not changed since the previous iteration of the
governor computations for that CPU, the CPU has been busy for all
that time and its frequency should not be decreased, so if the new
frequency would be lower than the one set previously, the governor
will skip the frequency update.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: Joel Fernandes <joelaf@google.com>
(cherry picked from commit b7eaf1aab9f8bd2e49fceed77ebc66c1b5800718)
(simple CPUFREQ_RT_DL vs CPUFREQ_DL usage conflicts)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Change-Id: I531ec02c052944ee07a904dc2a25c59948ee762b
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
The loop in sugov_next_freq_shared() contains an if block to skip the
loop for the current CPU. This turns out to be an unnecessary
conditional in the scheduler's hot-path for every CPU in the policy.
It would be better to drop the conditional and make the loop treat all
the CPUs in the same way. That would eliminate the need of calling
sugov_iowait_boost() at the top of the routine.
To keep the code optimized to return early if the current CPU has RT/DL
flags set, move the flags check to sugov_update_shared() instead in
order to avoid the function call entirely.
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
(cherry picked from commit cba1dfb57b94c234728b689d9b00d4267fa1a879)
(modified for SCHED_CPUFREQ_DL vs SCHED_CPUFREQ_RT)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Change-Id: Ie046fdc8eda46821356750edd0fb6f7d077af363
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
get_next_freq() uses sg_cpu only to get sg_policy, which the callers of
get_next_freq() already have. Pass sg_policy instead of sg_cpu to
get_next_freq(), to make it more efficient.
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
(cherry picked from commit 655cb1ebff4b7918fc560502c3297af2d3c7d114)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Change-Id: Ia210058da32930a6cdb18258aa679cd1a44a747e
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
This patch rectifies a comment present in sugov_irq_work() function to
follow proper grammar.
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
(cherry picked from commit d06e622d3d9206e6a2cc45a0f9a3256da8773ff4)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Change-Id: Iaf996445d411725639d511432cc424086892a146
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
Execute the irq-work specific initialization/exit code only when the
fast path isn't available.
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
(cherry picked from commit 21ef57297b15a49b0c4dd4e7135c1a08e9a29a1c)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Change-Id: Icfd68f455ef71846d799fcd2d8ec6aa1bf59573e
Signed-off-by: Quentin Perret <quentin.perret@arm.com>
Viresh Kumar
Ke Wang
Patrick Bellasi
Chris Redpath
Brendan Jackman
Peter Zijlstra
Vincent Guittot
Dietmar Eggemann
Quentin Perret
Abhilash Kesavan
Rafael J. Wysocki
Greg Kroah-Hartman
Wanpeng Li
Konstantin Khlebnikov
Leo Yan