commit 329f19bd636cfa181b3317ac0cb57f660991a27d Author: Ben Hutchings Date: Tue Jan 9 00:35:23 2018 +0000 Linux 3.16.53 commit 8eb0a71f15bd4e5b798c3fbe2f9932a69318a437 Author: Jiri Kosina Date: Wed Jan 3 15:20:04 2018 +0100 kaiser: x86: Fix NMI handling On Mon, 4 Dec 2017, Hugh Dickins wrote: > kaiser-3.18.72.tar Hi Hugh, this hunk from 0024-kaiser-merged-update.patch: - SWITCH_KERNEL_CR3_NO_STACK + /* + * percpu variables are mapped with user CR3, so no need + * to switch CR3 here. + */ cld movq %rsp, %rdx movq PER_CPU_VAR(kernel_stack), %rsp is problematic, as the patchset actually never user-maps kernel_stack percpu variable, and therefore crashes on NMIs. The patch below is needed to make NMIs work properly. Signed-off-by: Ben Hutchings commit 6ea3e97f1db2adeb219fcdc451ab6ec85f6eb0ad Author: Kees Cook Date: Wed Jan 3 10:18:01 2018 -0800 KPTI: Report when enabled Make sure dmesg reports when KPTI is enabled. Signed-off-by: Kees Cook Signed-off-by: Greg Kroah-Hartman [bwh: Backported to 3.16: adjust context] Signed-off-by: Ben Hutchings commit 88c38c3fedd878e608e0eb6a90a74d3ee11ae696 Author: Kees Cook Date: Thu Jan 4 01:14:24 2018 +0000 KPTI: Rename to PAGE_TABLE_ISOLATION This renames CONFIG_KAISER to CONFIG_PAGE_TABLE_ISOLATION. Signed-off-by: Kees Cook Signed-off-by: Greg Kroah-Hartman [bwh: Backported to 3.16] Signed-off-by: Ben Hutchings commit c8c9f9fe74b2402c24221ebe7a3be3e48d500870 Author: Borislav Petkov Date: Mon Dec 25 13:57:16 2017 +0100 x86/kaiser: Move feature detection up ... before the first use of kaiser_enabled as otherwise funky things happen: about to get started... (XEN) d0v0 Unhandled page fault fault/trap [#14, ec=0000] (XEN) Pagetable walk from ffff88022a449090: (XEN) L4[0x110] = 0000000229e0e067 0000000000001e0e (XEN) L3[0x008] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S: fault at ffff82d08033fd08 entry.o#create_bounce_frame+0x135/0x14d (XEN) Domain 0 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-4.9.1_02-3.21 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e033:[] (XEN) RFLAGS: 0000000000000286 EM: 1 CONTEXT: pv guest (d0v0) Signed-off-by: Borislav Petkov Signed-off-by: Greg Kroah-Hartman [bwh: Backported to 3.16: adjust context] Signed-off-by: Ben Hutchings commit e36d606e53a8d4cac574cbb22b006f9129b88f5f Author: Jiri Kosina Date: Tue Jan 2 14:19:49 2018 +0100 kaiser: disabled on Xen PV Kaiser cannot be used on paravirtualized MMUs (namely reading and writing CR3). This does not work with KAISER as the CR3 switch from and to user space PGD would require to map the whole XEN_PV machinery into both. More importantly, enabling KAISER on Xen PV doesn't make too much sense, as PV guests use distinct %cr3 values for kernel and user already. Signed-off-by: Jiri Kosina Signed-off-by: Greg Kroah-Hartman Cc: Hugh Dickins [bwh: Backported to 3.16: use xen_pv_domain()] Signed-off-by: Ben Hutchings commit 1f30b9849d6a59834d7f9f11e232e52958a1fbb7 Author: Borislav Petkov Date: Tue Jan 2 14:19:49 2018 +0100 x86/kaiser: Reenable PARAVIRT Now that the required bits have been addressed, reenable PARAVIRT. Signed-off-by: Borislav Petkov Signed-off-by: Greg Kroah-Hartman Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 33c7732274af0330804480d9472b26a67047f078 Author: Thomas Gleixner Date: Mon Dec 4 15:07:30 2017 +0100 x86/paravirt: Dont patch flush_tlb_single commit a035795499ca1c2bd1928808d1a156eda1420383 upstream. native_flush_tlb_single() will be changed with the upcoming PAGE_TABLE_ISOLATION feature. This requires to have more code in there than INVLPG. Remove the paravirt patching for it. Signed-off-by: Thomas Gleixner Reviewed-by: Josh Poimboeuf Reviewed-by: Juergen Gross Acked-by: Peter Zijlstra Cc: Andy Lutomirski Cc: Boris Ostrovsky Cc: Borislav Petkov Cc: Borislav Petkov Cc: Brian Gerst Cc: Dave Hansen Cc: Dave Hansen Cc: David Laight Cc: Denys Vlasenko Cc: Eduardo Valentin Cc: Greg KH Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Rik van Riel Cc: Will Deacon Cc: aliguori@amazon.com Cc: daniel.gruss@iaik.tugraz.at Cc: hughd@google.com Cc: keescook@google.com Cc: linux-mm@kvack.org Cc: michael.schwarz@iaik.tugraz.at Cc: moritz.lipp@iaik.tugraz.at Cc: richard.fellner@student.tugraz.at Link: https://lkml.kernel.org/r/20171204150606.828111617@linutronix.de Signed-off-by: Ingo Molnar Cc: Hugh Dickins [bwh: Backported to 3.16: adjust context] Signed-off-by: Ben Hutchings commit ef940cac4eb4714f8fa93f8ae857ebe1508df42e Author: Hugh Dickins Date: Sat Nov 4 18:43:06 2017 -0700 kaiser: kaiser_flush_tlb_on_return_to_user() check PCID Let kaiser_flush_tlb_on_return_to_user() do the X86_FEATURE_PCID check, instead of each caller doing it inline first: nobody needs to optimize for the noPCID case, it's clearer this way, and better suits later changes. Replace those no-op X86_CR3_PCID_KERN_FLUSH lines by a BUILD_BUG_ON() in load_new_mm_cr3(), in case something changes. (cherry picked from Change-Id: I9b528ed9d7c1ae4a3b4738c2894ee1740b6fb0b9) Signed-off-by: Hugh Dickins Signed-off-by: Ben Hutchings commit 6a3fe2c11b0c3686ba6cdfc57dcaf4759d6fe595 Author: Hugh Dickins Date: Sat Nov 4 18:23:24 2017 -0700 kaiser: asm/tlbflush.h handle noPGE at lower level I found asm/tlbflush.h too twisty, and think it safer not to avoid __native_flush_tlb_global_irq_disabled() in the kaiser_enabled case, but instead let it handle kaiser_enabled along with cr3: it can just use __native_flush_tlb() for that, no harm in re-disabling preemption. (This is not the same change as Kirill and Dave have suggested for upstream, flipping PGE in cr4: that's neat, but needs a cpu_has_pge check; cr3 is enough for kaiser, and thought to be cheaper than cr4.) Also delete the X86_FEATURE_INVPCID invpcid_flush_all_nonglobals() preference from __native_flush_tlb(): unlike the invpcid_flush_all() preference in __native_flush_tlb_global(), it's not seen in upstream 4.14, and was recently reported to be surprisingly slow. (cherry picked from Change-Id: I0da819a797ff46bca6590040b6480178dff6ba1e) Signed-off-by: Hugh Dickins Signed-off-by: Ben Hutchings commit 379ec248223b343872f71ec502dbdbf05e064818 Author: Hugh Dickins Date: Tue Oct 3 20:49:04 2017 -0700 kaiser: use ALTERNATIVE instead of x86_cr3_pcid_noflush Now that we're playing the ALTERNATIVE game, use that more efficient method: instead of user-mapping an extra page, and reading an extra cacheline each time for x86_cr3_pcid_noflush. Neel has found that __stringify(bts $X86_CR3_PCID_NOFLUSH_BIT, %rax) is a working substitute for the "bts $63, %rax" in these ALTERNATIVEs; but the one line with $63 in looks clearer, so let's stick with that. Worried about what happens with an ALTERNATIVE between the jump and jump label in another ALTERNATIVE? I was, but have checked the combinations in SWITCH_KERNEL_CR3_NO_STACK at entry_SYSCALL_64, and it does a good job. (cherry picked from Change-Id: I46d06167615aa8d628eed9972125ab2faca93f05) Signed-off-by: Hugh Dickins Signed-off-by: Ben Hutchings commit 58370efdb9e21815ecfadd12f4073a9584a431f2 Author: Borislav Petkov Date: Tue Jan 2 14:19:48 2018 +0100 x86/kaiser: Check boottime cmdline params AMD (and possibly other vendors) are not affected by the leak KAISER is protecting against. Keep the "nopti" for traditional reasons and add pti= like upstream. Signed-off-by: Borislav Petkov Signed-off-by: Greg Kroah-Hartman Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 4a86410f47b8c6e134fd9eeb8808007fa54de956 Author: Borislav Petkov Date: Tue Jan 2 14:19:48 2018 +0100 x86/kaiser: Rename and simplify X86_FEATURE_KAISER handling Concentrate it in arch/x86/mm/kaiser.c and use the upstream string "nopti". Signed-off-by: Borislav Petkov Signed-off-by: Greg Kroah-Hartman Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 07caf29fedd1e0c479da233fd094fb7a9b9a2c88 Author: Tom Lendacky Date: Mon Jul 17 16:10:33 2017 -0500 x86/boot: Add early cmdline parsing for options with arguments commit e505371dd83963caae1a37ead9524e8d997341be upstream. Add a cmdline_find_option() function to look for cmdline options that take arguments. The argument is returned in a supplied buffer and the argument length (regardless of whether it fits in the supplied buffer) is returned, with -1 indicating not found. Signed-off-by: Tom Lendacky Reviewed-by: Thomas Gleixner Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Arnd Bergmann Cc: Borislav Petkov Cc: Brijesh Singh Cc: Dave Young Cc: Dmitry Vyukov Cc: Jonathan Corbet Cc: Konrad Rzeszutek Wilk Cc: Larry Woodman Cc: Linus Torvalds Cc: Matt Fleming Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Peter Zijlstra Cc: Radim Krčmář Cc: Rik van Riel Cc: Toshimitsu Kani Cc: kasan-dev@googlegroups.com Cc: kvm@vger.kernel.org Cc: linux-arch@vger.kernel.org Cc: linux-doc@vger.kernel.org Cc: linux-efi@vger.kernel.org Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/36b5f97492a9745dce27682305f990fc20e5cf8a.1500319216.git.thomas.lendacky@amd.com Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 3c5ae93e51fbde6964271275b32e55bb91821f43 Author: Dave Hansen Date: Tue Dec 22 14:52:43 2015 -0800 x86/boot: Pass in size to early cmdline parsing commit 8c0517759a1a100a8b83134cf3c7f254774aaeba upstream. We will use this in a few patches to implement tests for early parsing. Signed-off-by: Dave Hansen [ Aligned args properly. ] Signed-off-by: Borislav Petkov Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: fenghua.yu@intel.com Cc: yu-cheng.yu@intel.com Link: http://lkml.kernel.org/r/20151222225243.5CC47EB6@viggo.jf.intel.com Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit bc74db22344982e0e9ff0bcf12e24a42f2fa96bd Author: Dave Hansen Date: Tue Dec 22 14:52:41 2015 -0800 x86/boot: Simplify early command line parsing commit 4de07ea481361b08fe13735004dafae862482d38 upstream. __cmdline_find_option_bool() tries to account for both NULL-terminated and non-NULL-terminated strings. It keeps 'pos' to look for the end of the buffer and also looks for '!c' in a bunch of places to look for NULL termination. But, it also calls strlen(). You can't call strlen on a non-NULL-terminated string. If !strlen(cmdline), then cmdline[0]=='\0'. In that case, we will go in to the while() loop, set c='\0', hit st_wordstart, notice !c, and will immediately return 0. So, remove the strlen(). It is unnecessary and unsafe. Signed-off-by: Dave Hansen Signed-off-by: Borislav Petkov Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: fenghua.yu@intel.com Cc: yu-cheng.yu@intel.com Link: http://lkml.kernel.org/r/20151222225241.15365E43@viggo.jf.intel.com Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit bfb3d23bb7bf7a436ce25c74e12e8710a4ecca90 Author: Dave Hansen Date: Tue Dec 22 14:52:39 2015 -0800 x86/boot: Fix early command-line parsing when partial word matches commit abcdc1c694fa4055323cbec1cde4c2cb6b68398c upstream. cmdline_find_option_bool() keeps track of position in two strings: 1. the command-line 2. the option we are searchign for in the command-line We plow through each character in the command-line one at a time, always moving forward. We move forward in the option ('opptr') when we match characters in 'cmdline'. We reset the 'opptr' only when we go in to the 'st_wordstart' state. But, if we fail to match an option because we see a space (state=st_wordcmp, *opptr='\0',c=' '), we set state='st_wordskip' and 'break', moving to the next character. But, that move to the next character is the one *after* the ' '. This means that we will miss a 'st_wordstart' state. For instance, if we have cmdline = "foo fool"; and are searching for "fool", we have: "fool" opptr = ----^ "foo fool" c = --------^ We see that 'l' != ' ', set state=st_wordskip, break, and then move 'c', so: "foo fool" c = ---------^ and are still in state=st_wordskip. We will stay in wordskip until we have skipped "fool", thus missing the option we were looking for. This *only* happens when you have a partially- matching word followed by a matching one. To fix this, we always fall *into* the 'st_wordskip' state when we set it. Signed-off-by: Dave Hansen Signed-off-by: Borislav Petkov Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: fenghua.yu@intel.com Cc: yu-cheng.yu@intel.com Link: http://lkml.kernel.org/r/20151222225239.8E1DCA58@viggo.jf.intel.com Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit f1bd7f13f5107cdfe035b246ca4ce52211b08a86 Author: Dave Hansen Date: Tue Dec 22 14:52:38 2015 -0800 x86/boot: Fix early command-line parsing when matching at end commit 02afeaae9843733a39cd9b11053748b2d1dc5ae7 upstream. The x86 early command line parsing in cmdline_find_option_bool() is buggy. If it matches a specified 'option' all the way to the end of the command-line, it will consider it a match. For instance, cmdline = "foo"; cmdline_find_option_bool(cmdline, "fool"); will return 1. This is particularly annoying since we have actual FPU options like "noxsave" and "noxsaves" So, command-line "foo bar noxsave" will match *BOTH* a "noxsave" and "noxsaves". (This turns out not to be an actual problem because "noxsave" implies "noxsaves", but it's still confusing.) To fix this, we simplify the code and stop tracking 'len'. 'len' was trying to indicate either the NULL terminator *OR* the end of a non-NULL-terminated command line at 'COMMAND_LINE_SIZE'. But, each of the three states is *already* checking 'cmdline' for a NULL terminator. We _only_ need to check if we have overrun 'COMMAND_LINE_SIZE', and that we can do without keeping 'len' around. Also add some commends to clarify what is going on. Signed-off-by: Dave Hansen Signed-off-by: Borislav Petkov Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: fenghua.yu@intel.com Cc: yu-cheng.yu@intel.com Link: http://lkml.kernel.org/r/20151222225238.9AEB560C@viggo.jf.intel.com Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 5a322c0ca6ad629240d0a1f19304160ef2d31a91 Author: Hugh Dickins Date: Sun Sep 24 16:59:49 2017 -0700 kaiser: add "nokaiser" boot option, using ALTERNATIVE Added "nokaiser" boot option: an early param like "noinvpcid". Most places now check int kaiser_enabled (#defined 0 when not CONFIG_KAISER) instead of #ifdef CONFIG_KAISER; but entry_64.S and entry_64_compat.S are using the ALTERNATIVE technique, which patches in the preferred instructions at runtime. That technique is tied to x86 cpu features, so X86_FEATURE_KAISER fabricated ("" in its comment so "kaiser" not magicked into /proc/cpuinfo). Prior to "nokaiser", Kaiser #defined _PAGE_GLOBAL 0: revert that, but be careful with both _PAGE_GLOBAL and CR4.PGE: setting them when nokaiser like when !CONFIG_KAISER, but not setting either when kaiser - neither matters on its own, but it's hard to be sure that _PAGE_GLOBAL won't get set in some obscure corner, or something add PGE into CR4. By omitting _PAGE_GLOBAL from __supported_pte_mask when kaiser_enabled, all page table setup which uses pte_pfn() masks it out of the ptes. It's slightly shameful that the same declaration versus definition of kaiser_enabled appears in not one, not two, but in three header files (asm/kaiser.h, asm/pgtable.h, asm/tlbflush.h). I felt safer that way, than with #including any of those in any of the others; and did not feel it worth an asm/kaiser_enabled.h - kernel/cpu/common.c includes them all, so we shall hear about it if they get out of synch. Cleanups while in the area: removed the silly #ifdef CONFIG_KAISER from kaiser.c; removed the unused native_get_normal_pgd(); removed the spurious reg clutter from SWITCH_*_CR3 macro stubs; corrected some comments. But more interestingly, set CR4.PSE in secondary_startup_64: the manual is clear that it does not matter whether it's 0 or 1 when 4-level-pts are enabled, but I was distracted to find cr4 different on BSP and auxiliaries - BSP alone was adding PSE, in probe_page_size_mask(). (cherry picked from Change-Id: I8e5bec716944444359cbd19f6729311eff943e9a) Signed-off-by: Hugh Dickins [bwh: Backported to 3.16: adjust context] Signed-off-by: Ben Hutchings commit 376ba2d81c95a66c018e0b0f71b16b8921714e56 Author: Borislav Petkov Date: Sat Jan 10 20:34:07 2015 +0100 x86/alternatives: Use optimized NOPs for padding commit 4fd4b6e5537cec5b56db0b22546dd439ebb26830 upstream. Alternatives allow now for an empty old instruction. In this case we go and pad the space with NOPs at assembly time. However, there are the optimal, longer NOPs which should be used. Do that at patching time by adding alt_instr.padlen-sized NOPs at the old instruction address. Cc: Andy Lutomirski Signed-off-by: Borislav Petkov Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 8f6482cd4c3594ccc2394f69a4bfe08c1d6f26e9 Author: Borislav Petkov Date: Mon Jan 5 13:48:41 2015 +0100 x86/alternatives: Make JMPs more robust commit 48c7a2509f9e237d8465399d9cdfe487d3212a23 upstream. Up until now we had to pay attention to relative JMPs in alternatives about how their relative offset gets computed so that the jump target is still correct. Or, as it is the case for near CALLs (opcode e8), we still have to go and readjust the offset at patching time. What is more, the static_cpu_has_safe() facility had to forcefully generate 5-byte JMPs since we couldn't rely on the compiler to generate properly sized ones so we had to force the longest ones. Worse than that, sometimes it would generate a replacement JMP which is longer than the original one, thus overwriting the beginning of the next instruction at patching time. So, in order to alleviate all that and make using JMPs more straight-forward we go and pad the original instruction in an alternative block with NOPs at build time, should the replacement(s) be longer. This way, alternatives users shouldn't pay special attention so that original and replacement instruction sizes are fine but the assembler would simply add padding where needed and not do anything otherwise. As a second aspect, we go and recompute JMPs at patching time so that we can try to make 5-byte JMPs into two-byte ones if possible. If not, we still have to recompute the offsets as the replacement JMP gets put far away in the .altinstr_replacement section leading to a wrong offset if copied verbatim. For example, on a locally generated kernel image old insn VA: 0xffffffff810014bd, CPU feat: X86_FEATURE_ALWAYS, size: 2 __switch_to: ffffffff810014bd: eb 21 jmp ffffffff810014e0 repl insn: size: 5 ffffffff81d0b23c: e9 b1 62 2f ff jmpq ffffffff810014f2 gets corrected to a 2-byte JMP: apply_alternatives: feat: 3*32+21, old: (ffffffff810014bd, len: 2), repl: (ffffffff81d0b23c, len: 5) alt_insn: e9 b1 62 2f ff recompute_jumps: next_rip: ffffffff81d0b241, tgt_rip: ffffffff810014f2, new_displ: 0x00000033, ret len: 2 converted to: eb 33 90 90 90 and a 5-byte JMP: old insn VA: 0xffffffff81001516, CPU feat: X86_FEATURE_ALWAYS, size: 2 __switch_to: ffffffff81001516: eb 30 jmp ffffffff81001548 repl insn: size: 5 ffffffff81d0b241: e9 10 63 2f ff jmpq ffffffff81001556 gets shortened into a two-byte one: apply_alternatives: feat: 3*32+21, old: (ffffffff81001516, len: 2), repl: (ffffffff81d0b241, len: 5) alt_insn: e9 10 63 2f ff recompute_jumps: next_rip: ffffffff81d0b246, tgt_rip: ffffffff81001556, new_displ: 0x0000003e, ret len: 2 converted to: eb 3e 90 90 90 ... and so on. This leads to a net win of around 40ish replacements * 3 bytes savings =~ 120 bytes of I$ on an AMD guest which means some savings of precious instruction cache bandwidth. The padding to the shorter 2-byte JMPs are single-byte NOPs which on smart microarchitectures means discarding NOPs at decode time and thus freeing up execution bandwidth. Signed-off-by: Borislav Petkov Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 005db1f8ff3e424440cf4746a1af6cc3dbdf699a Author: Borislav Petkov Date: Sat Dec 27 10:41:52 2014 +0100 x86/alternatives: Add instruction padding commit 4332195c5615bf748624094ce4ff6797e475024d upstream. Up until now we have always paid attention to make sure the length of the new instruction replacing the old one is at least less or equal to the length of the old instruction. If the new instruction is longer, at the time it replaces the old instruction it will overwrite the beginning of the next instruction in the kernel image and cause your pants to catch fire. So instead of having to pay attention, teach the alternatives framework to pad shorter old instructions with NOPs at buildtime - but only in the case when len(old instruction(s)) < len(new instruction(s)) and add nothing in the >= case. (In that case we do add_nops() when patching). This way the alternatives user shouldn't have to care about instruction sizes and simply use the macros. Add asm ALTERNATIVE* flavor macros too, while at it. Also, we need to save the pad length in a separate struct alt_instr member for NOP optimization and the way to do that reliably is to carry the pad length instead of trying to detect whether we're looking at single-byte NOPs or at pathological instruction offsets like e9 90 90 90 90, for example, which is a valid instruction. Thanks to Michael Matz for the great help with toolchain questions. Signed-off-by: Borislav Petkov Cc: Hugh Dickins [bwh: Backported to 3.16: adjust context] Signed-off-by: Ben Hutchings commit 6d1dd74880cc8aa019f21f4736448d6e004473e9 Author: Borislav Petkov Date: Tue Dec 30 20:27:09 2014 +0100 x86/alternatives: Cleanup DPRINTK macro commit db477a3386dee183130916d6bbf21f5828b0b2e2 upstream. Make it pass __func__ implicitly. Also, dump info about each replacing we're doing. Fixup comments and style while at it. Signed-off-by: Borislav Petkov Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit a606925a28b389f1c726851fe35f07917ffaeafe Author: Hugh Dickins Date: Sun Dec 17 19:53:01 2017 -0800 kaiser: alloc_ldt_struct() use get_zeroed_page() Change the 3.2.96 and 3.18.72 alloc_ldt_struct() to allocate its entries with get_zeroed_page(), as 4.3 onwards does since f454b4788613 ("x86/ldt: Fix small LDT allocation for Xen"). This then matches the free_page() I had misported in __free_ldt_struct(), and fixes the "BUG: Bad page state in process ldt_gdt_32 ... flags: 0x80(slab)" reported by Kees Cook and Jiri Kosina, and analysed by Jiri. Signed-off-by: Hugh Dickins Signed-off-by: Ben Hutchings commit 082345aaa10a01ef140240a1d9f864a3cc17436e Author: Ben Hutchings Date: Fri Jan 5 03:09:26 2018 +0000 x86: kvmclock: Disable use from vDSO if KPTI is enabled Currently the pvclock pages aren't being added to user-space page tables, and my attempt to fix this didn't work. Signed-off-by: Ben Hutchings commit 7b31cc057cc19f414d20f4c5c0d3b252441b8742 Author: Borislav Petkov Date: Thu Jan 4 17:42:45 2018 +0100 kaiser: Set _PAGE_USER of the vsyscall page Signed-off-by: Borislav Petkov Signed-off-by: Hugh Dickins [bwh: Backported to 3.16: - Drop the case for disabled CONFIG_X86_VSYSCALL_EMULATION - Adjust filename, context] Signed-off-by: Ben Hutchings commit f9a1666f97b32836058839ab03f49daef0528ca0 Author: Richard Fellner Date: Thu May 4 14:26:50 2017 +0200 KAISER: Kernel Address Isolation This patch introduces our implementation of KAISER (Kernel Address Isolation to have Side-channels Efficiently Removed), a kernel isolation technique to close hardware side channels on kernel address information. More information about the patch can be found on: https://github.com/IAIK/KAISER From: Richard Fellner From: Daniel Gruss Subject: [RFC, PATCH] x86_64: KAISER - do not map kernel in user mode Date: Thu, 4 May 2017 14:26:50 +0200 Link: http://marc.info/?l=linux-kernel&m=149390087310405&w=2 Kaiser-4.10-SHA1: c4b1831d44c6144d3762ccc72f0c4e71a0c713e5 To: To: Cc: Cc: Cc: Michael Schwarz Cc: Richard Fellner Cc: Ingo Molnar Cc: Cc: After several recent works [1,2,3] KASLR on x86_64 was basically considered dead by many researchers. We have been working on an efficient but effective fix for this problem and found that not mapping the kernel space when running in user mode is the solution to this problem [4] (the corresponding paper [5] will be presented at ESSoS17). With this RFC patch we allow anybody to configure their kernel with the flag CONFIG_KAISER to add our defense mechanism. If there are any questions we would love to answer them. We also appreciate any comments! Cheers, Daniel (+ the KAISER team from Graz University of Technology) [1] http://www.ieee-security.org/TC/SP2013/papers/4977a191.pdf [2] https://www.blackhat.com/docs/us-16/materials/us-16-Fogh-Using-Undocumented-CPU-Behaviour-To-See-Into-Kernel-Mode-And-Break-KASLR-In-The-Process.pdf [3] https://www.blackhat.com/docs/us-16/materials/us-16-Jang-Breaking-Kernel-Address-Space-Layout-Randomization-KASLR-With-Intel-TSX.pdf [4] https://github.com/IAIK/KAISER [5] https://gruss.cc/files/kaiser.pdf (cherry picked from Change-Id: I0eb000c33290af01fc4454ca0c701d00f1d30b1d) Conflicts: arch/x86/entry/entry_64.S (not in this tree) arch/x86/kernel/entry_64.S (patched instead of that) arch/x86/entry/entry_64_compat.S (not in this tree) arch/x86/ia32/ia32entry.S (patched instead of that) arch/x86/include/asm/hw_irq.h arch/x86/include/asm/pgtable_types.h arch/x86/include/asm/processor.h arch/x86/kernel/irqinit.c arch/x86/kernel/process.c arch/x86/mm/Makefile arch/x86/mm/pgtable.c init/main.c Signed-off-by: Hugh Dickins [bwh: Folded in the follow-up patches from Hugh: - kaiser: merged update - kaiser: do not set _PAGE_NX on pgd_none - kaiser: stack map PAGE_SIZE at THREAD_SIZE-PAGE_SIZE - kaiser: fix build and FIXME in alloc_ldt_struct() - kaiser: KAISER depends on SMP - kaiser: fix regs to do_nmi() ifndef CONFIG_KAISER - kaiser: fix perf crashes - kaiser: ENOMEM if kaiser_pagetable_walk() NULL - kaiser: tidied up asm/kaiser.h somewhat - kaiser: tidied up kaiser_add/remove_mapping slightly - kaiser: kaiser_remove_mapping() move along the pgd - kaiser: align addition to x86/mm/Makefile - kaiser: cleanups while trying for gold link - kaiser: name that 0x1000 KAISER_SHADOW_PGD_OFFSET - kaiser: delete KAISER_REAL_SWITCH option - kaiser: vmstat show NR_KAISERTABLE as nr_overhead - kaiser: enhanced by kernel and user PCIDs - kaiser: load_new_mm_cr3() let SWITCH_USER_CR3 flush user - kaiser: PCID 0 for kernel and 128 for user - kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user - kaiser: paranoid_entry pass cr3 need to paranoid_exit - kaiser: _pgd_alloc() without __GFP_REPEAT to avoid stalls - kaiser: fix unlikely error in alloc_ldt_struct() - kaiser: drop is_atomic arg to kaiser_pagetable_walk() Backported to 3.16: - Add missing #include in arch/x86/mm/kaiser.c - Use variable PEBS buffer size since we have "perf/x86/intel: Use PAGE_SIZE for PEBS buffer size on Core2" - Renumber X86_FEATURE_INVPCID_SINGLE to avoid collision - Adjust context] Signed-off-by: Ben Hutchings commit 7ebf71f82595bc72f9308d7bb193367cd6534f25 Author: Andy Lutomirski Date: Sun Oct 8 21:53:05 2017 -0700 x86/mm/64: Fix reboot interaction with CR4.PCIDE commit 924c6b900cfdf376b07bccfd80e62b21914f8a5a upstream. Trying to reboot via real mode fails with PCID on: long mode cannot be exited while CR4.PCIDE is set. (No, I have no idea why, but the SDM and actual CPUs are in agreement here.) The result is a GPF and a hang instead of a reboot. I didn't catch this in testing because neither my computer nor my VM reboots this way. I can trigger it with reboot=bios, though. Fixes: 660da7c9228f ("x86/mm: Enable CR4.PCIDE on supported systems") Reported-and-tested-by: Steven Rostedt (VMware) Signed-off-by: Andy Lutomirski Signed-off-by: Thomas Gleixner Cc: Borislav Petkov Link: https://lkml.kernel.org/r/f1e7d965998018450a7a70c2823873686a8b21c0.1507524746.git.luto@kernel.org Cc: Hugh Dickins [bwh: Backported to 3.16: use clear_in_cr4()] Signed-off-by: Ben Hutchings commit 7e031825d38f43a6c2a5b1010c2c087e57bfc853 Author: Andy Lutomirski Date: Thu Jun 29 08:53:21 2017 -0700 x86/mm: Enable CR4.PCIDE on supported systems commit 660da7c9228f685b2ebe664f9fd69aaddcc420b5 upstream. We can use PCID if the CPU has PCID and PGE and we're not on Xen. By itself, this has no effect. A followup patch will start using PCID. Signed-off-by: Andy Lutomirski Reviewed-by: Nadav Amit Reviewed-by: Boris Ostrovsky Reviewed-by: Thomas Gleixner Cc: Andrew Morton Cc: Arjan van de Ven Cc: Borislav Petkov Cc: Dave Hansen Cc: Juergen Gross Cc: Linus Torvalds Cc: Mel Gorman Cc: Peter Zijlstra Cc: Rik van Riel Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/6327ecd907b32f79d5aa0d466f04503bbec5df88.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar [Hugh Dickins: Backported to 3.18: - arch/x86/xen/enlighten_pv.c (not in this tree) - arch/x86/xen/enlighten.c (patched instead of that)] Signed-off-by: Hugh Dickins [Borislav Petkov: Fix bad backport to disable PCID on Xen] Signed-off-by: Ben Hutchings commit 18daed7ec73db97b2adaae2f43232cc33c95d2de Author: Andy Lutomirski Date: Thu Jun 29 08:53:20 2017 -0700 x86/mm: Add the 'nopcid' boot option to turn off PCID commit 0790c9aad84901ca1bdc14746175549c8b5da215 upstream. The parameter is only present on x86_64 systems to save a few bytes, as PCID is always disabled on x86_32. Signed-off-by: Andy Lutomirski Reviewed-by: Nadav Amit Reviewed-by: Borislav Petkov Reviewed-by: Thomas Gleixner Cc: Andrew Morton Cc: Arjan van de Ven Cc: Borislav Petkov Cc: Dave Hansen Cc: Linus Torvalds Cc: Mel Gorman Cc: Peter Zijlstra Cc: Rik van Riel Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/8bbb2e65bcd249a5f18bfb8128b4689f08ac2b60.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar [Hugh Dickins: Backported to 3.18: - Documentation/admin-guide/kernel-parameters.txt (not in this tree) - Documentation/kernel-parameters.txt (patched instead of that) Signed-off-by: Hugh Dickins Signed-off-by: Ben Hutchings commit 5623a10f06c4bdeea6080a0dd721ef62b95f9ff4 Author: Andy Lutomirski Date: Thu Jun 29 08:53:19 2017 -0700 x86/mm: Disable PCID on 32-bit kernels commit cba4671af7550e008f7a7835f06df0763825bf3e upstream. 32-bit kernels on new hardware will see PCID in CPUID, but PCID can only be used in 64-bit mode. Rather than making all PCID code conditional, just disable the feature on 32-bit builds. Signed-off-by: Andy Lutomirski Reviewed-by: Nadav Amit Reviewed-by: Borislav Petkov Reviewed-by: Thomas Gleixner Cc: Andrew Morton Cc: Arjan van de Ven Cc: Borislav Petkov Cc: Dave Hansen Cc: Linus Torvalds Cc: Mel Gorman Cc: Peter Zijlstra Cc: Rik van Riel Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/2e391769192a4d31b808410c383c6bf0734bc6ea.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar Signed-off-by: Hugh Dickins Signed-off-by: Ben Hutchings commit a98ff222bda022ba66c490378f4069f112514919 Author: Andy Lutomirski Date: Sun May 28 10:00:14 2017 -0700 x86/mm: Remove the UP asm/tlbflush.h code, always use the (formerly) SMP code commit ce4a4e565f5264909a18c733b864c3f74467f69e upstream. The UP asm/tlbflush.h generates somewhat nicer code than the SMP version. Aside from that, it's fallen quite a bit behind the SMP code: - flush_tlb_mm_range() didn't flush individual pages if the range was small. - The lazy TLB code was much weaker. This usually wouldn't matter, but, if a kernel thread flushed its lazy "active_mm" more than once (due to reclaim or similar), it wouldn't be unlazied and would instead pointlessly flush repeatedly. - Tracepoints were missing. Aside from that, simply having the UP code around was a maintanence burden, since it means that any change to the TLB flush code had to make sure not to break it. Simplify everything by deleting the UP code. Signed-off-by: Andy Lutomirski Cc: Andrew Morton Cc: Arjan van de Ven Cc: Borislav Petkov Cc: Dave Hansen Cc: Linus Torvalds Cc: Mel Gorman Cc: Michal Hocko Cc: Nadav Amit Cc: Nadav Amit Cc: Peter Zijlstra Cc: Rik van Riel Cc: Thomas Gleixner Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar [Hugh Dickins: Backported to 3.18] Signed-off-by: Hugh Dickins Signed-off-by: Ben Hutchings commit 9748818e52ff2617d8720e2586e5b3d976540da5 Author: Andy Lutomirski Date: Mon May 22 15:30:01 2017 -0700 x86/mm: Reimplement flush_tlb_page() using flush_tlb_mm_range() commit ca6c99c0794875c6d1db6e22f246699691ab7e6b upstream. flush_tlb_page() was very similar to flush_tlb_mm_range() except that it had a couple of issues: - It was missing an smp_mb() in the case where current->active_mm != mm. (This is a longstanding bug reported by Nadav Amit) - It was missing tracepoints and vm counter updates. The only reason that I can see for keeping it at as a separate function is that it could avoid a few branches that flush_tlb_mm_range() needs to decide to flush just one page. This hardly seems worthwhile. If we decide we want to get rid of those branches again, a better way would be to introduce an __flush_tlb_mm_range() helper and make both flush_tlb_page() and flush_tlb_mm_range() use it. Signed-off-by: Andy Lutomirski Acked-by: Kees Cook Cc: Andrew Morton Cc: Borislav Petkov Cc: Dave Hansen Cc: Linus Torvalds Cc: Mel Gorman Cc: Michal Hocko Cc: Nadav Amit Cc: Nadav Amit Cc: Peter Zijlstra Cc: Rik van Riel Cc: Thomas Gleixner Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/3cc3847cf888d8907577569b8bac3f01992ef8f9.1495492063.git.luto@kernel.org Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 62ad49f5649e325e81ea6aaa1d52a475310c7fec Author: Andy Lutomirski Date: Sat Apr 22 00:01:21 2017 -0700 x86/mm: Make flush_tlb_mm_range() more predictable commit ce27374fabf553153c3f53efcaa9bfab9216bd8c upstream. I'm about to rewrite the function almost completely, but first I want to get a functional change out of the way. Currently, if flush_tlb_mm_range() does not flush the local TLB at all, it will never do individual page flushes on remote CPUs. This seems to be an accident, and preserving it will be awkward. Let's change it first so that any regressions in the rewrite will be easier to bisect and so that the rewrite can attempt to change no visible behavior at all. The fix is simple: we can simply avoid short-circuiting the calculation of base_pages_to_flush. As a side effect, this also eliminates a potential corner case: if tlb_single_page_flush_ceiling == TLB_FLUSH_ALL, flush_tlb_mm_range() could have ended up flushing the entire address space one page at a time. Signed-off-by: Andy Lutomirski Acked-by: Dave Hansen Cc: Andrew Morton Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Josh Poimboeuf Cc: Linus Torvalds Cc: Michal Hocko Cc: Nadav Amit Cc: Peter Zijlstra Cc: Rik van Riel Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/4b29b771d9975aad7154c314534fec235618175a.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit b7ebb3d8c097026998bb72c017c1e25fe2882a55 Author: Andy Lutomirski Date: Sat Apr 22 00:01:20 2017 -0700 x86/mm: Remove flush_tlb() and flush_tlb_current_task() commit 29961b59a51f8c6838a26a45e871a7ed6771809b upstream. I was trying to figure out what how flush_tlb_current_task() would possibly work correctly if current->mm != current->active_mm, but I realized I could spare myself the effort: it has no callers except the unused flush_tlb() macro. Signed-off-by: Andy Lutomirski Cc: Andrew Morton Cc: Borislav Petkov Cc: Brian Gerst Cc: Dave Hansen Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Josh Poimboeuf Cc: Linus Torvalds Cc: Michal Hocko Cc: Nadav Amit Cc: Peter Zijlstra Cc: Rik van Riel Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/e52d64c11690f85e9f1d69d7b48cc2269cd2e94b.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar Cc: Hugh Dickins [bwh: Backported to 3.16: adjust context] Signed-off-by: Ben Hutchings commit b1f76e6a5010f926cc276c161493afb8939fadf9 Author: Andy Lutomirski Date: Sat Apr 22 00:01:19 2017 -0700 x86/vm86/32: Switch to flush_tlb_mm_range() in mark_screen_rdonly() commit 9ccee2373f0658f234727700e619df097ba57023 upstream. mark_screen_rdonly() is the last remaining caller of flush_tlb(). flush_tlb_mm_range() is potentially faster and isn't obsolete. Compile-tested only because I don't know whether software that uses this mechanism even exists. Signed-off-by: Andy Lutomirski Cc: Andrew Morton Cc: Borislav Petkov Cc: Brian Gerst Cc: Dave Hansen Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Josh Poimboeuf Cc: Linus Torvalds Cc: Michal Hocko Cc: Nadav Amit Cc: Peter Zijlstra Cc: Rik van Riel Cc: Sasha Levin Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/791a644076fc3577ba7f7b7cafd643cc089baa7d.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 34c2165d1e4f29257372b796af3d81271fe8c3c7 Author: Aaron Lu Date: Thu Aug 11 15:44:30 2016 +0800 x86/irq: Do not substract irq_tlb_count from irq_call_count commit 82ba4faca1bffad429f15c90c980ffd010366c25 upstream. Since commit: 52aec3308db8 ("x86/tlb: replace INVALIDATE_TLB_VECTOR by CALL_FUNCTION_VECTOR") the TLB remote shootdown is done through call function vector. That commit didn't take care of irq_tlb_count, which a later commit: fd0f5869724f ("x86: Distinguish TLB shootdown interrupts from other functions call interrupts") ... tried to fix. The fix assumes every increase of irq_tlb_count has a corresponding increase of irq_call_count. So the irq_call_count is always bigger than irq_tlb_count and we could substract irq_tlb_count from irq_call_count. Unfortunately this is not true for the smp_call_function_single() case. The IPI is only sent if the target CPU's call_single_queue is empty when adding a csd into it in generic_exec_single. That means if two threads are both adding flush tlb csds to the same CPU's call_single_queue, only one IPI is sent. In other words, the irq_call_count is incremented by 1 but irq_tlb_count is incremented by 2. Over time, irq_tlb_count will be bigger than irq_call_count and the substract will produce a very large irq_call_count value due to overflow. Considering that: 1) it's not worth to send more IPIs for the sake of accurate counting of irq_call_count in generic_exec_single(); 2) it's not easy to tell if the call function interrupt is for TLB shootdown in __smp_call_function_single_interrupt(). Not to exclude TLB shootdown from call function count seems to be the simplest fix and this patch just does that. This bug was found by LKP's cyclic performance regression tracking recently with the vm-scalability test suite. I have bisected to commit: 3dec0ba0be6a ("mm/rmap: share the i_mmap_rwsem") This commit didn't do anything wrong but revealed the irq_call_count problem. IIUC, the commit makes rwc->remap_one in rmap_walk_file concurrent with multiple threads. When remap_one is try_to_unmap_one(), then multiple threads could queue flush TLB to the same CPU but only one IPI will be sent. Since the commit was added in Linux v3.19, the counting problem only shows up from v3.19 onwards. Signed-off-by: Aaron Lu Cc: Alex Shi Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Davidlohr Bueso Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Huang Ying Cc: Josh Poimboeuf Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Tomoki Sekiyama Link: http://lkml.kernel.org/r/20160811074430.GA18163@aaronlu.sh.intel.com Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit b90eacdcdf0787c043ce5b76f2db1bd17df5ba95 Author: Andy Lutomirski Date: Fri Jun 9 11:49:15 2017 -0700 sched/core: Idle_task_exit() shouldn't use switch_mm_irqs_off() commit 252d2a4117bc181b287eeddf848863788da733ae upstream. idle_task_exit() can be called with IRQs on x86 on and therefore should use switch_mm(), not switch_mm_irqs_off(). This doesn't seem to cause any problems right now, but it will confuse my upcoming TLB flush changes. Nonetheless, I think it should be backported because it's trivial. There won't be any meaningful performance impact because idle_task_exit() is only used when offlining a CPU. Signed-off-by: Andy Lutomirski Cc: Borislav Petkov Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: stable@vger.kernel.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/ca3d1a9fa93a0b49f5a8ff729eda3640fb6abdf9.1497034141.git.luto@kernel.org Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 9e81e8e222899e2d8d9802c77af82ffc7012c9f0 Author: Steven Rostedt Date: Fri May 13 15:30:13 2016 +0200 ARM: Hide finish_arch_post_lock_switch() from modules commit ef0491ea17f8019821c7e9c8e801184ecf17f85a upstream. The introduction of switch_mm_irqs_off() brought back an old bug regarding the use of preempt_enable_no_resched: As part of: 62b94a08da1b ("sched/preempt: Take away preempt_enable_no_resched() from modules") the definition of preempt_enable_no_resched() is only available in built-in code, not in loadable modules, so we can't generally use it from header files. However, the ARM version of finish_arch_post_lock_switch() calls preempt_enable_no_resched() and is defined as a static inline function in asm/mmu_context.h. This in turn means we cannot include asm/mmu_context.h from modules. With today's tip tree, asm/mmu_context.h gets included from linux/mmu_context.h, which is normally the exact pattern one would expect, but unfortunately, linux/mmu_context.h can be included from the vhost driver that is a loadable module, now causing this compile time error with modular configs: In file included from ../include/linux/mmu_context.h:4:0, from ../drivers/vhost/vhost.c:18: ../arch/arm/include/asm/mmu_context.h: In function 'finish_arch_post_lock_switch': ../arch/arm/include/asm/mmu_context.h:88:3: error: implicit declaration of function 'preempt_enable_no_resched' [-Werror=implicit-function-declaration] preempt_enable_no_resched(); Andy already tried to fix the bug by including linux/preempt.h from asm/mmu_context.h, but that didn't help. Arnd suggested reordering the header files, which wasn't popular, so let's use this workaround instead: The finish_arch_post_lock_switch() definition is now also hidden inside of #ifdef MODULE, so we don't see anything referencing preempt_enable_no_resched() from a header file. I've built a few hundred randconfig kernels with this, and did not see any new problems. Tested-by: Guenter Roeck Signed-off-by: Steven Rostedt Signed-off-by: Arnd Bergmann Acked-by: Russell King Cc: Alexander Shishkin Cc: Andy Lutomirski Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnaldo Carvalho de Melo Cc: Borislav Petkov Cc: Frederic Weisbecker Cc: Jiri Olsa Cc: Linus Torvalds Cc: Mel Gorman Cc: Peter Zijlstra Cc: Russell King - ARM Linux Cc: Stephane Eranian Cc: Thomas Gleixner Cc: Vince Weaver Cc: linux-arm-kernel@lists.infradead.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/1463146234-161304-1-git-send-email-arnd@arndb.de Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 691795e9ea88914b6a7d1eeaabffa0959e2bb283 Author: Andy Lutomirski Date: Tue Apr 26 09:39:09 2016 -0700 x86/mm, sched/core: Turn off IRQs in switch_mm() commit 078194f8e9fe3cf54c8fd8bded48a1db5bd8eb8a upstream. Potential races between switch_mm() and TLB-flush or LDT-flush IPIs could be very messy. AFAICT the code is currently okay, whether by accident or by careful design, but enabling PCID will make it considerably more complicated and will no longer be obviously safe. Fix it with a big hammer: run switch_mm() with IRQs off. To avoid a performance hit in the scheduler, we take advantage of our knowledge that the scheduler already has IRQs disabled when it calls switch_mm(). Signed-off-by: Andy Lutomirski Reviewed-by: Borislav Petkov Cc: Borislav Petkov Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/f19baf759693c9dcae64bbff76189db77cb13398.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar Cc: Hugh Dickins Signed-off-by: Ben Hutchings commit 7a2d99e974a300c772208f00d9f1a2f8b052bfe3 Author: Andy Lutomirski Date: Tue Apr 26 09:39:08 2016 -0700 x86/mm, sched/core: Uninline switch_mm() commit 69c0319aabba45bcf33178916a2f06967b4adede upstream. It's fairly large and it has quite a few callers. This may also help untangle some headers down the road. Signed-off-by: Andy Lutomirski Reviewed-by: Borislav Petkov